https://orcid.org/0000-0001-9314-5281
Figures
Abstract
Objective
Despite the available evidence regarding effectiveness of stroke telerehabilitation, there has been little focus on factors influencing its delivery or translation from the research setting into practice. There are complex challenges to embedding telerehabilitation into stroke services and generating transferable knowledge about scaling up and routinising this service model. This review aimed to explore factors influencing the delivery of stroke telerehabilitation interventions, including platforms, technical requirements, training, support, access, cost, usability and acceptability.
Methods
MEDLINE, EMBASE, CINAHL, Web of Science and Cochrane Library and Central Registry of Clinical Trials were searched to identify full-text articles of randomised controlled trials (RCTs) and protocols for RCTs published since a Cochrane review on stroke telerehabilitation services. A narrative synthesis was conducted, providing a comprehensive description of the factors influencing stroke telerehabilitation intervention delivery.
Results
Thirty-one studies and ten protocols of ongoing studies were included. Interventions were categorised as synchronous telerehabilitation (n = 9), asynchronous telerehabilitation (n = 11) and tele-support (n = 11). Telephone and videoconference were the most frequently used modes of delivery. Usability and acceptability with telerehabilitation were high across all platforms, although access issues and technical challenges may be potential barriers to the use of telerehabilitation in service delivery. Costs of intervention delivery and training requirements were poorly reported.
Conclusions
This review synthesises the evidence relating to factors that may influence stroke telerehabilitation intervention delivery at a crucial timepoint given the rapid deployment of telerehabilitation in response to the COVID-19 pandemic. It recommends strategies, such as ensuring adequate training and technical infrastructure, shared learning and consistent reporting of cost and usability and acceptability outcomes, to overcome challenges in embedding and routinising this service model and priorities for research in this area.
Citation: Stephenson A, Howes S, Murphy PJ, Deutsch JE, Stokes M, Pedlow K, et al. (2022) Factors influencing the delivery of telerehabilitation for stroke: A systematic review. PLoS ONE 17(5): e0265828. https://doi.org/10.1371/journal.pone.0265828
Editor: Amir-Homayoun Javadi, University of Kent, UNITED KINGDOM
Received: April 28, 2021; Accepted: March 8, 2022; Published: May 11, 2022
Copyright: © 2022 Stephenson et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting information files.
Funding: AS’s time was co-funded by Royal College of Surgeons in Ireland (School of Physiotherapy) and the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement no. 687228, MAGIC PCP. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Rehabilitation is one of the most important aspects of care following a stroke, leading to better recovery and higher levels of independence [1]. Globally the prevalence of stroke has increased by 85% in the last thirty years, and it now represents the condition with the highest need for rehabilitation worldwide [2]. This means that increasingly, despite positive evidence for post stroke rehabilitation [1, 3], the recommended amount of therapy is rarely available or achieved [4] resulting in unmet ongoing rehabilitation needs [2, 5]. This limitation on therapy is further compounded by the COVID-19 pandemic, causing widespread disruption of healthcare services and concern that healthcare facilities could be sources of contagion [6, 7]. Given the disease transmission mechanism and requirement to reduce in-person contacts, including between patient and clinician, telerehabilitation offers a unique solution allowing convenient access to post-stroke rehabilitation without exposure risk [6, 7]. It has been recommended to prevent service interruption, where quarantine or social distancing measures have been advised [8, 9]. In addition to its current necessity in response to COVID-19, telehealth may continue to contribute to the solution to longstanding limitations on therapy. It may free up clinician time and address some barriers faced by people with stroke such as time restraints, geographical isolation and compliance [10].
Telerehabilitation is a branch of telehealth including the provision of rehabilitation services to patients at a remote location using information and communication technologies across distance or time [11, 12]. Several recent reviews supporting telerehabilitation for stroke rehabilitation compared to in-person care, are centred around clinical effectiveness [10, 13–15]. Despite the available evidence regarding the effectiveness of telerehabilitation, there has been little focus on factors influencing telerehabilitation delivery or its translation from the research setting into stroke practice, including technical requirements, challenges, practicalities, and factors related to usability and acceptability. The latter two are known factors that impact on digital intervention uptake and continued use [16]. Given the varied degree of impairments and activity limitations experienced post-stroke, for example impacting motor function, cognitive function and communication [3], additional considerations may be required for telerehabilitation post-stroke to ensure accessibility and engagement. Given the opaque timeline of COVID-19 and future longer-term disruptions to stroke rehabilitation services, it is crucial that these factors are explored. Given the well-established evidence, the intention of this systematic review is not to provide definitive conclusions regarding the effectiveness of stroke telerehabilitation. The intention is to search and synthesise the evidence regarding the practicalities of delivering telerehabilitation in stroke care. This will help design appropriate interventions and identify factors to be considered to enable stroke telerehabilitation reach its full potential.
Aims and objectives
The aim was to identify and describe the scientific literature relating to factors influencing the delivery of stroke telerehabilitation interventions. The specific objectives included:
- To synthesise intervention delivery, including platforms used, dose and technical requirements.
- To summarise training and support requirements for intervention delivery.
- To explore factors relating to access of telerehabilitation in this clinical area and the cost of delivering telerehabilitation.
- To explore the usability and acceptability of stroke telerehabilitation interventions, including participant-reported outcomes, adherence, adverse events, and facilitators and barriers to use.
Methods
The protocol was developed a priori according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and registered on Prospero (CRD42020186024). The registered protocol uses the term rapid review; however, on reflection, this is a systematic review given the comprehensive data extraction and synthesis.
Data sources and searches
Studies were initially identified from a recent Cochrane review on telerehabilitation services for stroke [13] which identified papers up to December 2018. This was supplemented by our search of five electronic bibliographic databases (MEDLINE, EMBASE, CINAHL, Web of Science and Cochrane Library and Central Registry of Clinical Trials) between January 2019 to May 2020, to identify papers published since the Cochrane review [13].
Predefined search strategies, based on those used in the Cochrane review [13], were developed with assistance of a librarian and piloted prior to use. The Medline search strategy can be found in S1 File.
Reference lists of eligible studies were hand-searched, citation tracking of these publications conducted and a Google Scholar search performed to identify additional studies missed in the original searches.
The search results were imported into ProQuest RefWorks bibliographic software and duplicate studies removed. Screening was divided amongst the reviewers using the Covidence systematic review software. Titles and abstracts and full text papers of potentially relevant studies were screened by two independent reviewers. Conflicts were decided by an independent verifier.
Study selection
This review included full-text articles of randomised controlled trials (RCTs) and protocols for RCTs published in English, that delivered telerehabilitation interventions to people with stroke.
This review included adult stroke survivors with all types of stroke, at all levels of severity, and at all stages post-stroke (acute, subacute, or chronic). It excluded studies involving a mixture of stroke and non-stroke participants where data about stroke participants was not reported separately. Trials with children were excluded given the low stroke incidence and the additional challenges to delivering therapy via technological means in this paediatric population.
For the purposes of this review, telerehabilitation was defined as the provision of rehabilitation services, including assessment, review or rehabilitation, to patients at a remote location using information and communication technologies [11]. Interventions where telerehabilitation was not a major component were excluded, judged by team consensus, e.g. intervention included only one telerehabilitation session; the participants received more in-person than telerehabilitation contact; the only telerehabilitation component was either automated, not monitored by clinician/researcher, or only a helpline if required.
There were no restrictions related to clinical outcomes or context, as we were interested in interventions performed in all settings and geographical locations, and delivered by all types of therapists or non-therapists.
Data extraction and quality assessment
Study details and data were extracted using a customised form, developed based on an eHealth checklist [17], and piloted prior to use (S2 File). The form was used to capture information related to demographics, intervention/control arm details, outcomes and results. Telerehabilitation intervention delivery, such as platforms used, dose and technical requirements were extracted. Human support and training related to the delivery of telerehabilitation required for participants, their carers and clinicians delivering the telerehabilitation was extracted. We extracted data related to access, such as relevant eligibility criteria and requirements for inclusion in the studies, and costs. Usability and acceptability data extracted consisted of participant-reported outcomes, including data related to usability, acceptability and satisfaction from the patient, carer or clinician perspective; adherence-related outcomes, such as usage of systems, completion of sessions and engagement with rehabilitation; safety and adverse events; and facilitators and barriers to use. Data extraction was completed independently for each paper by one of two reviewers (50% AS, 50%SH), with 20% checked by another reviewer (10% KP, 10% SM).
Given the range of telehealth approaches being used across the studies, the research team agreed the following definitions: synchronous telerehabilitation was used to describe interventions with real-time clinician-patient interaction during real-time review of the rehabilitation activity; asynchronous telerehabilitation was used to describe interventions where rehabilitation activity was conducted independently by the patient and their progress reviewed later by the therapist with a follow-up clinician-patient interaction to review rehabilitation progress; and, telesupport was used to describe interventions that provided patients only with support, advice or education related to their stroke. Technical support or helplines were not categorised as telesupport, for these purposes. Where interventions delivered more than one type of telerehabilitation, they were categorised based on the greatest component of the intervention. Definitions and categorisation according to definitions were agreed by consensus within the research team.
Risk of bias was assessed (for completed studies only) by a single reviewer, verified by a second. Where agreement could not be achieved with discussion, a third reviewer completed a consensus assessment. We assessed each study using the Cochrane Risk of Bias tool (V1) [18] for consistency with Laver [13], grading on each criterion as having low, high, or unclear risk of bias. A study was judged to be at low risk of bias overall when all domains had a low risk of bias. Conversely, a study was judged to have a high risk of bias when it reported a feature judged as high risk of bias in any domain.
Data synthesis and analysis
A narrative synthesis was conducted, providing a comprehensive description of the telerehabilitation interventions for stroke rehabilitation. The synthesis explored factors relating to delivery of the interventions, such as platforms used and technical requirements, training and support required, access and costs and other facilitators and challenges to implementation. The synthesis described patterns in adherence, usability and acceptability and explored factors that may contribute to any differences across the included studies.
Results
The PRISMA flow diagram (Fig 1) summarises the study selection process. The search strategy identified 2092 results. An additional 266 records were identified by hand searching. Of these, the full texts of 159 studies were screened for inclusion. This review included 31 studies [19–49] and ten protocols of ongoing studies [50–59], adding eleven studies and five protocols to the recent Cochrane review [13].
See Table 1 for study characteristics of included RCTs, which included a total of 3368 participants, ranging from 10 to 573 participants, with 58% male. The majority of studies were conducted in the USA (n = 8).
Telerehabilitation interventions
Intervention delivery.
Comprehensive descriptions of the interventions can be found in S3 File. Table 2 provides a summary of telerehabilitation intervention characteristics. The rehabilitation aims of the interventions and type of telerehabilitation delivered varied. Most interventions were aimed at improving a stroke primary or secondary impairment: physical function (n = 10); upper limb function (n = 5); speech and language ability (n = 3); cognitive function (n = 2); and visual impairment (n = 1). Others targeted self-management, including secondary prevention and health behaviour change (n = 4), quality of life (n = 4) and mental health (n = 2).
The interventions that aimed to improve a stroke primary or secondary impairment (n = 21) were delivered by asynchronous telerehabilitation, where the participant completed self-led therapy which was reviewed by remote consultation with the clinician (n = 10), real-time synchronous telerehabilitation, where the clinician remotely supervised the participants’ therapy in real-time (n = 9), or telesupport, where remote consultations were for education, support, or goal-setting only (n = 2).
One intervention for self-management delivered asynchronous telerehabilitation via a mobile app and online chat [28]. The remaining interventions aimed at self-management, quality of life and mental health, were delivered by telesupport only.
Platforms used.
Most studies used either telephone (n = 15) or video (n = 15) call as a mode of communication within the telerehabilitation interventions; one of which compared videoconference versus telephone delivery [32]. Nine interventions involved telephone only [20, 21, 30, 31, 36, 42, 43, 47]; these tended to be interventions delivering telesupport. Telesupport was also delivered via online communication, such as email and online forum [44] or text reminder system plus telephone call [48].
The remaining n = 22 interventions included a combination of telerehabilitation components. Videoconference along with a digital component, such as a computer-, tablet- or app-based component, was used in n = 11 studies [19, 22, 23, 25, 26, 29, 33, 35, 37, 39, 40]. Telephone along with a digital component was used in n = 5 studies [24, 34, 45, 48, 49].
Dose.
Intervention duration was similar in interventions delivered by synchronous telerehabilitation (range = 4–12 weeks), and asynchronous telerehabilitation (range = 10 days to 12 weeks). However, the frequency of contact tended to be higher with synchronous telerehabilitation (once/week to twice/day), compared with asynchronous telerehabilitation (once/week to twice/3 months). Telesupport interventions tended to be of longer duration; almost half were six months or longer [28, 30, 41–43], and patient-clinician interaction was less frequent in telesupport interventions (twice/week to five calls/18 months).
Technical requirements.
Videoconferencing was primarily delivered via a laptop or personal computer with a webcam, using a variety of videoconferencing software: Skype [26]; WebEx or VSee [37]; Cisco/Jabber Acano [38]; or, not reported [22, 29, 33]. One study used the We Chat mobile app [32] and one used a desktop videophone [27]. The remaining eight studies used bespoke software accessed via a computer [23, 25, 39, 40] or a tablet [19, 35, 46] with videoconferencing capabilities.
Of the synchronous and asynchronous telerehabilitation interventions, clinicians were able to review the participants’ performance remotely in all but two studies [22, 45], in which data could not be accessed until the end of the intervention period. There was limited reporting on the software used for data collection, storage and transfer, which often appeared to be integrated within bespoke systems [23, 25, 33]. One study reported using the MySQL relational database management system [19]. Only one study reported using encrypted videoconference software [38], while the remaining studies did not report measures taken to ensure data protection/information security.
Training and support
Telerehabilitation delivery.
Clinicians delivered the telerehabilitation in over 80% of studies (26/31). Less than 10% was delivered by researchers, with a combination of clinician and researcher delivery in one study [24].
Clinician training.
Fewer than 20% of studies delivered by clinicians mentioned clinicians’ training (5/26).
Participant and/or carer training and support.
Just over 50% of studies reported participant and/or carer training details (Table 2). No studies provided direct links to their training materials or manuals. Two studies [20, 31] referred to contacting the corresponding authors for a copy of their manuals, but no response was received at the time of submission.
Access and costs
Access to telerehabilitation.
Study eligibility criteria excluded individuals with cognitive impairment (n = 21/31) and communication difficulties (n = 10/31). Two studies reported including additional measures to include participants with cognitive or communication difficulties, such as mailing their questionnaires if they had communication difficulties [42, 44].
The majority of studies appeared to provide study equipment, while just over 20% of studies (n = 5/22, excluding the telephone-only interventions), excluded participants who did not have access and/or the ability to operate a smartphone [32, 48], tablet [37], computer [49], internet access [29]. In one study, if participants could not access teleconferencing software at home, they attended a telehealth centre or received treatment in a separate room at the clinic with no direct contact with the clinician [37].
Costs of delivering interventions.
Cost analysis was reported in n = 2 studies. A balance gaming intervention costed 44% less to deliver remotely ($835.61) than in-person ($1490.23) [34]. Another study suggested that telephone follow-up had a 68% chance of being cost-saving in terms of health service utilisation, following an estimated a mean cost-saving of £311 in the telerehabilitation group compared with a usual care control [42].
Usability and acceptability
Participant-reported outcomes.
Participant-reported outcomes of usability and acceptability were reported in 35% of studies (n = 11/31), and frequently measured using quantitative measures, of which three studies included validated tools [25, 34, 44], or qualitative feedback (Table 1). In general, studies reported high levels of usability and acceptability, most frequently reported as user satisfaction, with telerehabilitation regardless of the different modes of delivery and platforms. Only one study measured the clinicians’ perspective [29], with findings that both participants and therapists were satisfied with the system despite dissatisfaction with some of the system aesthetics and the difficulty of the tasks.
Only one study compared two different modes of telerehabilitation delivery and reported that participants reported higher satisfaction and confidence using videoconference compared with telephone call for assessment of their functional status [32]. When compared with an in-person in-clinic control or usual care, participants reported comparably high levels of satisfaction with videoconference in combination with computer game-based therapy for upper limb rehabilitation [25, 29, 39] and balance training [33]. Participants also reported high levels of usability with telephone review of a computer game-based therapy for balance training [34], compared with in-clinic use of the system. In studies where telerehabilitation provided additional contact or support, such as telephone contact compared with usual care [21, 42] or online support compared with information only [44], participants tended to be more satisfied in the telerehabilitation group.
Participants receiving a multi-component intervention to improve physical activity levels including a mobile app and an in-person group rehabilitation program reported high levels of satisfaction, but many reported preferring the combination of the app and exercise program compared with either the app or exercise program alone [28].
Adherence.
Only 25% of studies (8/31) reported adherence-related outcomes [19, 25, 26, 28, 29, 32, 42, 49]. Reporting of adherence varied across studies (Table 1); for example, some studies reported the percentage of sessions completed [25, 32, 42], others reported the percentage of participants that completed all the sessions [26] or the average treatment time [19, 29], and not all studies reported adherence for the control groups.
In general, high levels of adherence were observed and, where applicable, adherence was comparable in telerehabilitation and control groups and was consistently at least 80% [25, 32, 49]; although in one study, low compliance with the smartphone app component of a multicomponent intervention (50% of participants used the app) was explained by difficulties using the technology [28].
Adverse events.
Only just over 20% of studies reported on adverse events, with n = 5 having no adverse events and n = 2 having expected adverse events such as upper limb pain [25] and fatigue [25, 26]. Incidence and type of adverse events were comparable in the intervention and control groups.
Facilitators and barriers to use.
In general, telerehabilitation offered participants increased opportunity for therapy [29, 46]. Positive feedback for telerehabilitation for stroke included improved access to, and interaction with, therapists [19, 48] and participants appreciated the telerehabilitation contact provided [20, 41]. Telephone reviews were more accessible and less disruptive to daily routine than in-person care [31, 42].
Technology-related barriers included: telephone tag [20]; internet connectivity issues [25, 28, 39]; the availability of the technology required [24, 28, 37]; equipment costs [33, 34]; difficulties using the device [28]; the need for additional training or support [24, 45]; and dissatisfaction with the aesthetics of the system [29]. One study added that assistance for technical support was required less frequently as participants progressed through the intervention [25].
Wariness of technology was a barrier to recruitment in one study where n = 21 individuals declined participation due to concerns about use of mobile health technology [32]; although n = 120 participants were recruited, and dropouts were similar (≤20%) in both videoconference and telephone call groups.
Published protocols of ongoing trials
Study characteristics of included protocols are summarised in Table 3. Table 4 provides a summary of telerehabilitation intervention characteristics from the protocols, with comprehensive descriptions of the interventions to be delivered found in S4 File.
Telerehabilitation interventions
Intervention delivery.
The protocols (n = 10) identified suggest that ongoing trials are investigating telerehabilitation interventions targeting physical function (n = 5), upper limb rehabilitation (n = 2) and self-management including health behaviour and secondary prevention (n = 3). Interventions continue to centre around the use of telephone (n = 5) or videoconference (n = 4); one of which uses both telephone and video call in the telerehabilitation group [53]. However, two studies include telerehabilitation contact via an online platform only [51, 56]. Six protocols describe telesupport interventions, four protocols describe asynchronous telerehabilitation interventions, while none describe synchronous telerehabilitation. Most studies are being conducted in Canada (n = 3). See Table 3.
All interventions include a combination of intervention components, such as computer game-based therapy reviewed via videoconference [50, 55] or telephone [58]. Other intervention components that will be used in combination with video or telephone review, such as activity monitors used only by participants to self-monitor their physical activity [52, 57] or upper limb movement [55], and access to online or smartphone-enabled educational content [53, 59], were not reported on here, as they will not be reviewed remotely by the clinician.
Intervention durations ranged from three weeks to six months (Table 4); all the 6-month interventions are categorised as telecoaching. Frequency of telerehabilitation contact varies across the protocols from three times/week to contact twice/month.
Training and support.
Seven protocols describe participant training (Table 4). The level of support anticipated includes additional clinician support in n = 2 protocols, carer support in n = 6 protocols and technical support in n = 5 protocols. Two protocols state they will deliver clinician training [53, 57] (S3 File).
Access and costs.
As observed in the RCTs, the eligibility criteria have not substantially changed, in that people with communication or cognitive challenges continue to be excluded. Similarly, the majority of studies continue to provide the necessary equipment.
Usability and acceptability.
Increased numbers of protocols are planning to measure adherence (60%), with all telerehabilitation systems having capability to monitor usage. Measurement of usability and acceptability outcomes remains low (40%) with plans to measure via qualitative feedback [51, 53] and self-developed [53, 58] and validated questionnaires [50, 51, 58].
Risk of bias of included studies
Risk of bias of included studies is summarised in S5 File. The additional studies (n = 11) showed a similar risk of bias pattern to the studies included in the Cochrane review [13]. Three studies were judged to have high risk of bias [28, 35, 46]. Six studies were judged to be at unclear risk of bias [19, 30, 32, 38, 45, 49]. One study was judged to be at a low risk of bias [42].
Discussion
To our knowledge, this is the first review to focus on synthesising the evidence for stroke telerehabilitation intervention delivery relating to factors that may influence its uptake and continued use. Given the available evidence related to its effectiveness, this review aimed to synthesise the evidence regarding the practicalities of delivering telerehabilitation in stroke care. There are complex challenges to embedding telerehabilitation into healthcare services and also generating transferable knowledge about scaling up and routinising this service model [60]. This work is essential as it informs the efforts needed to maintain stroke rehabilitation services in the era of Covid-19 and beyond.
Telerehabilitation intervention delivery
The studies included in this review mainly used videoconferencing or the telephone, either alone or in combination with other intervention components, to deliver stroke telerehabilitation. This review also suggests that that telerehabilitation most often sits as a component within a complex intervention, and its centrality to the intervention and the level of interaction with clinical teams vary. One of the challenges in the synthesis of our findings was the variations in how telerehabilitation was delivered and how to categorise these approaches. We used three categories or definitions of telerehabilitation e.g. synchronous and asynchronous telerehabilitation and telesupport. We searched four international physiotherapy organisations’ resources, and were unable to find standardised definitions [61–64], this echoes previous findings that there appears to be no universally agreed definition of telerehabilitation [10]. Use of standardised terminology to describe telerehabilitation delivery may help clinicians identify the most appropriate ways to include telerehabilitation within their services, find the most appropriate resources to support set-up of such services, and communicate effectively with patients about how their telerehabilitation will be delivered. This may improve both patient and clinician understanding and satisfaction, and could be addressed by bringing the community together to agree international consensus, for example in a Delphi study [65].
Given the rise in the popularity of more advanced technologies to deliver telerehabilitation for stroke, it is interesting to note that in terms of modes of delivery, the telephone has not been overlooked as a platform, due to its accessibility and simplicity of use. Alternatively, an advantage of videoconference identified by this review was its potential to facilitate synchronous telerehabilitation whereby participants received real-time clinician interaction and review of their rehabilitation. The only study that compared two telerehabilitation modes of delivery [32], found higher participant satisfaction and confidence with videoconference compared with telephone. Advice given for stroke care during the COVID-19 pandemic is that videoconference is superior to telephone, but telephone consultation is superior to no consultation [66]. Nonetheless, although the current COVID-19 pandemic has accelerated efforts to overcome technological challenges associated with telerehabilitation technologies, improved access to the required hardware and internet connection, and appropriate training and support are required for the implementation of videoconference clinically.
Training and support
Provision of appropriate training and ongoing support, for both the patients and clinicians, is essential to improve confidence and levels of usability and acceptability contributing to the adoption and maintenance of telerehabilitation in clinical practice [67, 68]. Reporting on training of people with stroke was good (>50% studies) but there was a notable omission of clinician training (20% studies). This is needed to ensure the sustainability of telehealth interventions in the health service and address the urgent and increasing need for stroke rehabilitation worldwide [2]. Additionally, limited detail was available on the content or delivery of the training. As telehealth continues to be used in the response to the pandemic, we are presented with an opportunity for shared learning. Access to these training materials and resources may have facilitated the rapid set-up of telehealth services required during the current COVID-19 pandemic and in the future. As well as sharing of training resources to support healthcare providers who are introducing remote service delivery and research teams investigating this service model, there may be value in documenting individual experience and wider context learning to prevent duplication and open discussion about optimal delivery. Establishing and sharing resources such as Standard Operating Procedures (SOPs), trouble shooting documents, creating contingency plans, training webinars and manuals would allow for the generation of a knowledge base. This would assist in guiding the processes around telerehabilitation, and may help overcome challenges of embedding such interventions in clinical practice [69].
Access and costs
In response to the pandemic, and even after the threat of COVID-19 subsides, the “new normal” for rehabilitation services is likely to include a certain amount of telerehabilitation [70–72]. With this shift, it is important that digital equality disparities regarding restricted access and/or low digital literacy are not exacerbated. Those without access to, or who could not use technology were not eligible to participate in over 20% of the studies in the current review. Additionally, study participants were frequently provided with the required equipment, which may not be feasible in practice. There is the risk of delivering less and lower-quality care to the most underserved, by allowing internet access and device ownership to become social determinants of health [60, 73, 74]. Efforts must be made to address the specific needs of those with low digital literacy so that the use of digital technologies does not risk excluding and further disadvantaging this population [75, 76]. Solutions to encourage equitable access may include: additional in-person appointments for those without access to the required equipment; flexibility of clinicians to use the equipment available to the patients, redistribution of refurbished devices; and, providing education, training and support to encourage digital literacy [74, 77].
The reported cost analyses suggest that telerehabilitation may provide a cost-effective alternative that may enable delivery of rehabilitation superior to usual care without the time or resource required for in-person rehabilitation. However, the reporting of intervention delivery costs (2/31 studies; 4/10 protocols) is insufficient to inform service providers implementing telerehabilitation in stroke care, and the lack of clear and consistent reporting of the methods used with insufficient detail to replicate. The need for improved reporting on cost-effectiveness of telerehabilitation has previously been reported [78, 79]. The lack of cost information available is surprising given that cost is often cited as a barrier to setting up telerehabilitation, but equally could be deemed cost-saving and efficient in the long run. Evaluation of cost-effectiveness comparing telerehabilitation and usual care costs, considering start-up costs, clinician time, travel and healthcare utilisation, should be prioritised and incorporated into future telerehabilitation research in a real-world context [13, 78, 79].
Usability and acceptability
Research on the usability and acceptability of telerehabilitation is essential to enhance uptake and sustainability of service delivery [16, 80, 81]. While these outcomes continue to be inconsistently reported, the findings of this review were encouraging in relation to usability and acceptability. Participant-reported usability and acceptability with telerehabilitation (reported in 11/31 studies of which three used validated outcome measures) was high across various platforms, comparable to in-person interaction and superior to inactive control arms, such as usual care. Additionally, the high levels of adherence to telerehabilitation interventions observed were comparable to in-person rehabilitation, and no safety concerns related to the delivery of telerehabilitation interventions were reported. While adherence in the completed trials was poorly (25% papers) and inconsistently reported, a greater proportion of the protocols for ongoing trials stated they will measure adherence (60%). Perceived benefits of telerehabilitation reported included improved accessibility to clinicians, increased opportunity for rehabilitation and time efficiency for both patient and clinician. This is particularly pertinent at present given the challenges for both patients accessing and service providers delivering rehabilitation due to the COVID-19 pandemic [REF]. Barriers reported were frequently related to ease of use, which was influenced by the difficulty of using the system as well as connectivity and technical issues. This resulted in low compliance with a smartphone app component of one intervention due to difficulty using it, and frequent requirement of additional support. To ensure successful translation of telerehabilitation into stroke management, it is necessary to ensure the benefits of telerehabilitation are not outweighed by technical challenges. Telerehabilitation service delivery should include easy to follow guidelines for use that are tailored to users’ functional abilities and preferences, the opportunity to trial or practice telerehabilitation during a familiarisation period, availability of ongoing technical support, and options for in-person appointments if required. Additionally, consistent reporting of usability- and acceptability-related outcomes throughout the pathway from research to practice, including the use of validated outcome measures and exploration of the factors influencing of usability and acceptability through patient and public involvement (PPI) and qualitative feedback, would facilitate optimisation of interventions to improve uptake and ongoing engagement with telerehabilitation in stroke care.
Limitations
To make the review process more efficient, we included studies from a Cochrane review with our database searches identifying additional papers published since its last search date [13]. It is also possible we missed relevant studies which were not published in English. Additionally, by including only published protocols, not all trial registrations, it is likely that there is additional ongoing research in this area. For efficiency, data extraction and risk of bias assessment were carried out by a single reviewer with verification by a second reviewer rather than by two independent reviewers. Additionally, all the studies included in this review were conducted prior to the pandemic outbreak. Therefore, it is likely that due to the rapid deployment of various digital devices to deliver contact-free care, that other methods to deliver telerehabilitation have and will continue to emerge across the literature.
Conclusion
Given the available evidence, the aim of this systematic review was not to provide definitive conclusions regarding the effectiveness of telerehabilitation, but to search and synthesise the evidence regarding the practicalities of integrating telerehabilitation in stroke care. The main findings of this review are that stroke telerehabilitation is generally usable and acceptable and can work well with appropriate training and technical infrastructure. This review recommends improved shared learning from service users and providers to optimise rehabilitation outcomes, patient experience and service quality. It highlighted the need for standardised terminology to describe telerehabilitation, potentially through a Delphi study. As telerehabilitation continues to be used in the response to the COVID-19 pandemic, this review highlighted the need for consistent reporting of the practicalities, challenges and costs of implementation of telerehabilitation into stroke care. This systematic review provides a deeper understanding of telerehabilitation delivery and its translation into stroke practice.
Supporting information
S3 File. Description of telerehabilitation interventions (papers).
https://doi.org/10.1371/journal.pone.0265828.s003
(DOCX)
S4 File. Description of telerehabilitation interventions (protocols).
https://doi.org/10.1371/journal.pone.0265828.s004
(DOCX)
References
- 1.
National Institute for Health and Care Excellence. Stroke rehabilitation: long-term rehabilitation after stroke (Clinical guideline CG 162). 2013. http://guidance.nice.org.uk/cg162. Accessed August 12, 2020.
- 2. Cieza A, Causey K, Kamenov K, Hanson SW, Chatterji S, Vos T. Global estimates of the need for rehabilitation based on the Global Burden of Disease study 2019: a systematic analysis for the Global Burden of Disease Study 2019. The Lancet. 2020;396:2006–17. pmid:33275908
- 3.
Bowen A, James M, Young G. Royal College of Physicians. National clinical guideline for stroke, 2016. RCP.
- 4.
Royal College of Physicians. Sentinel stroke national audit programme (SSNAP) Clinical audit July-Sept 2014 Public Report, 2015. https://www.strokeaudit.org/Documents/National/Clinical/JulSep2014/JulSep2014-PublicReport.aspx. Accessed August 12, 2020.
- 5.
Stroke Association. State of the Nation: Stroke Statistics, 2020. https://www.stroke.org.uk/resources/state-nation-stroke-statistics. Accessed August 12, 2020.
- 6. Hollander JE, Carr BG. Virtually perfect? Telemedicine for COVID-19. New England Journal of Medicine. 2020;382:1679–81. pmid:32160451
- 7. McIntyre M, Robinson LR, Mayo A. Practical Considerations for Implementing Virtual Care in Physical Medicine and Rehabilitation: For the Pandemic and Beyond. American Journal of Physical Medicine & Rehabilitation. 2020;99:464. pmid:32324617
- 8. Greenhalgh T, Wherton J, Shaw S, Morrison C. Video consultations for covid-19. BMJ 2020;368:m998. pmid:32165352
- 9.
HSE, 2020. Guidance For Setting Up Virtual Pulmonary Rehabilitation for Asthma And COPD During The Covid 19 Pandemic. [online] Hse.ie. <https://www.hse.ie/eng/about/who/cspd/ncps/copd/resources/ncp-respiratory-guidance-on-setting-up-virtual-pulmonary-rehabilitation-for-asthma-and-copd.pdf> [Accessed 8 July 2020].
- 10. Appleby E, Gill ST, Hayes LK, Walker TL, Walsh M, Kumar S. Effectiveness of telerehabilitation in the management of adults with stroke: A systematic review. PLOS one. 2019;14:e0225150. pmid:31714924
- 11. Brennan DM, Mawson S, Brownsell S. Telerehabilitation: enabling the remote delivery of healthcare, rehabilitation, and self management. Stud Health Technol Inform. 2009;145:48.
- 12.
Totten AM, Womack DM, Eden KB, et al. Telehealth: Mapping the Evidence for Patient Outcomes From Systematic Reviews [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2016. (Technical Briefs, No. 26.) https://www.ncbi.nlm.nih.gov/books/NBK379320/. Accessed January 18, 2021.
- 13. Laver KE, Adey-Wakeling Z, Crotty M, Lannin NA, George S, Sherrington C. Telerehabilitation services for stroke. Cochrane Database of Systematic Reviews. 2020;1. pmid:32002991
- 14. Chen J, Jin W, Zhang XX, Xu W, Liu XN, Ren CC. Telerehabilitation approaches for stroke patients: systematic review and meta-analysis of randomized controlled trials. Journal of Stroke and Cerebrovascular Diseases. 2015;24:2660–8. pmid:26483155
- 15. Sarfo FS, Ulasavets U, Opare-Sem OK, Ovbiagele B. Tele-rehabilitation after stroke: an updated systematic review of the literature. Journal of Stroke and Cerebrovascular Diseases. 2018;27:2306–18. pmid:29880211
- 16. Nawaz A, Skjaeret N, Helbostad JL, et al. Usability and acceptability of balance exergames in older adults: A scoping review. Health Informatics J. 2016;22:911–931. pmid:26303810
- 17. Eysenbach G, Consort-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. Journal of medical Internet research. 2011;13(4):e126. pmid:22209829
- 18.
Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. www.handbook.cochrane.org. Accessed on May 12, 2020.
- 19. Asano M, Tai BC, Yeo FY, Yen SC, Tay A, Ng YS, et al. Home-based tele-rehabilitation presents comparable positive impact on self-reported functional outcomes as usual care: The Singapore Tele-technology Aided Rehabilitation in Stroke randomised trial. Journal of Telemedicine and Telecare. 2019;1357633X19868905.
- 20. Bishop D, Miller I, Weiner D, Guilmette T, Mukand J, Feldmann E et al. Family Intervention: Telephone Tracking (FITT): A Pilot Stroke Outcome Study. Topics in Stroke Rehabilitation. 2014;21:S63–S74. pmid:24722045
- 21. Boter H. Multicenter Randomized Controlled Trial of an Outreach Nursing Support Program for Recently Discharged Stroke Patients. Stroke. 2004;35:2867–2872. pmid:15514186
- 22. Carey J, Durfee W, Bhatt E, Nagpal A, Weinstein S, Anderson K et al. Comparison of Finger Tracking Versus Simple Movement Training via Telerehabilitation to Alter Hand Function and Cortical Reorganization After Stroke. Neurorehabilitation and Neural Repair. 2007;21:216–232. pmid:17351083
- 23. Chen J, Jin W, Dong W, Jin Y, Qiao F, Zhou Y et al. Effects of Home-based Telesupervising Rehabilitation on Physical Function for Stroke Survivors with Hemiplegia. American Journal of Physical Medicine & Rehabilitation. 2017;96:152–160. pmid:27386808
- 24. Chumbler N, Quigley P, Li X, Morey M, Rose D, Sanford J et al. Effects of Telerehabilitation on Physical Function and Disability for Stroke Patients. Stroke. 2012;43:2168–2174. pmid:22627983
- 25. Cramer S, Dodakian L, Le V, See J, Augsburger R, McKenzie A et al. Efficacy of Home-Based Telerehabilitation vs In-Clinic Therapy for Adults After Stroke. JAMA Neurology. 2019;76:1079. pmid:31233135
- 26. Deng H, Durfee W, Nuckley D, Rheude B, Severson A, Skluzacek K et al. Complex Versus Simple Ankle Movement Training in Stroke Using Telerehabilitation: A Randomized Controlled Trial. Physical Therapy. 2012;92:197–209. pmid:22095209
- 27. Forducey P, Glueckauf R, Bergquist T, Maheu M, Yutsis M. Telehealth for persons with severe functional disabilities and their caregivers: Facilitating self-care management in the home setting. Psychological Services. 2012;9:144–162. pmid:22662729
- 28. Grau-Pellicer M., Lalanza J.F., Jovell-Fernández E. and Capdevila L. Impact of mHealth technology on adherence to healthy PA after stroke: a randomized study. Topics in stroke rehabilitation. 2020;27:354–368. pmid:31790639
- 29. Huijgen B, Vollenbroek-Hutten M, Zampolini M, Opisso E, Bernabeu M, Van Nieuwenhoven J et al. Feasibility of a home-based telerehabilitation system compared to usual care: arm/hand function in patients with stroke, traumatic brain injury and multiple sclerosis. Journal of Telemedicine and Telecare. 2008;14:249–256. pmid:18633000
- 30. Joubert J, Davis SM, Donnan GA, Levi C, Gonzales G, Joubert L, et al. ICARUSS: an effective model for risk factor management in stroke survivors. International Journal of Stroke. 2020;15:438–453. pmid:30785379
- 31. Kirkness C, Cain K, Becker K, Tirschwell D, Buzaitis A, Weisman P et al. Randomized trial of telephone versus in-person delivery of a brief psychosocial intervention in post-stroke depression. BMC Research Notes. 2017;10:500. pmid:29017589
- 32. Li L, Huang J, Wu J, Jiang C, Chen S, Xie G, et al. A Mobile Health App for the Collection of Functional Outcomes After Inpatient Stroke Rehabilitation: Pilot Randomized Controlled Trial. JMIR mHealth and uHealth. 2020;8:e17219. pmid:32401221
- 33. Lin KH, Chen CH, Chen YY, Huang WT, Lai JS, Yu SM, et al. Bidirectional and multi-user telerehabilitation system: clinical effect on balance, functional activity, and satisfaction in patients with chronic stroke living in long-term care facilities. Sensors. 2014;14:12451–66. pmid:25019632
- 34. Lloréns R, Noé E, Colomer C, Alcañiz M. Effectiveness, usability, and cost-benefit of a virtual reality–based telerehabilitation program for balance recovery after stroke: A randomized controlled trial. Archives of physical medicine and rehabilitation. 2015;96:418–25. pmid:25448245
- 35. Maresca G, Maggio MG, Latella D, Cannavò A, De Cola MC, Portaro S, et al. Toward Improving Poststroke Aphasia: A Pilot Study on the Growing Use of Telerehabilitation for the Continuity of Care. Journal of Stroke and Cerebrovascular Diseases. 2019;28:104303. pmid:31371144
- 36. Mayo NE, Nadeau L, Ahmed S, White C, Grad R, Huang A, et al. Bridging the gap: the effectiveness of teaming a stroke coordinator with patient’s personal physician on the outcome of stroke. Age and ageing. 2008;37:32–8. pmid:18006510
- 37. Meltzer JA, Baird AJ, Steele RD, Harvey SJ. Computer-based treatment of poststroke language disorders: A non-inferiority study of telerehabilitation compared to in-person service delivery. Aphasiology. 2018;32:290–311.
- 38. Øra HP, Kirmess M, Brady MC, Partee I, Hognestad RB, Johannessen BB, et al. The effect of augmented speech-language therapy delivered by telerehabilitation on poststroke aphasia—a pilot randomized controlled trial. Clinical Rehabilitation. 2020;34:369–81. pmid:31903800
- 39. Piron L, Turolla A, Tonin P, Piccione F, Lain L, Dam M. Satisfaction with care in post-stroke patients undergoing a telerehabilitation programme at home. Journal of telemedicine and telecare. 2008;14:257–60. pmid:18633001
- 40. Piron L, Turolla A, Agostini M, Zucconi C, Cortese F, Zampolini M, et al. Exercises for paretic upper limb after stroke: a combined virtual-reality and telemedicine approach. Journal of Rehabilitation Medicine. 2009;41:1016–20. pmid:19841835
- 41. Rochette A, Korner-Bitensky N, Bishop D, Teasell R, White CL, Bravo G, et al. The YOU CALL–WE CALL randomized clinical trial: impact of a multimodal support intervention aMer a mild stroke. Circulation: Cardiovascular Quality and Outcomes. 2013;6:674–9.
- 42. Rodgers H, Howel D, Bhattarai N, Cant R, Drummond A, Ford GA, et al. Evaluation of an Extended Stroke Rehabilitation Service (EXTRAS) A Randomized Controlled Trial and Economic Analysis. Stroke. 2019;50:3561–8. pmid:31637972
- 43. Saal S, Becker C, Lorenz S, Schubert M, Kuss O, Stang A, et al. Effect of a stroke support service in Germany: a randomized trial. Topics in Stroke Rehabilitation. 2015;22:429–36. pmid:25920942
- 44. Smith GC, Egbert N, Dellman-Jenkins M, Nanna K, Palmieri PA. Reducing depression in stroke survivors and their informal caregivers: A randomized clinical trial of a web-based intervention. Rehabilitation psychology. 2012;57:196. pmid:22946607
- 45. Svaerke KW, Omkvist KV, Havsteen IB, Christensen HK. Computer-Based Cognitive Rehabilitation in Patients with Visuospatial Neglect or Homonymous Hemianopia after Stroke. Journal of Stroke and Cerebrovascular Diseases. 2019;28:104356. pmid:31495672
- 46. Torrisi M, Maresca G, De Cola MC, Cannavò A, Sciarrone F, Silvestri G, et al. Using telerehabilitation to improve cognitive function in post-stroke survivors: is this the time for the continuity of care?. International Journal of Rehabilitation Research. 2019;42:344–51. pmid:31464812
- 47. Wan LH, Zhang XP, Mo MM, Xiong XN, Ou CL, You LM, et al. Effectiveness of goal-setting telephone follow-up on health behaviors of patients with ischemic stroke: a randomized controlled trial. Journal of Stroke and Cerebrovascular Diseases. 2016;25:2259–70. pmid:27371106
- 48. Wang MY, Shen MJ, Wan LH, Mo MM, Wu Z, Li LL, et al. Effects of a Comprehensive Reminder System Based on the Health Belief Model for Patients Who Have Had a Stroke on Health Behaviors, Blood Pressure, Disability, and Recurrence From Baseline to 6 Months: A Randomized Controlled Trial. Journal of Cardiovascular Nursing. 2020;35:156–64.
- 49. Withiel TD, Wong D, Ponsford JL, Cadilhac DA, New P, Mihaljcic T, et al. Comparing memory group training and computerized cognitive training for improving memory function following stroke: A phase II randomized controlled trial. Journal of rehabilitation medicine. 2019;51:25–33. pmid:30815708
- 50. Allegue DR, Kairy D, Higgins J, Archambault P, Michaud F, Miller W, et al. Optimization of Upper Extremity Rehabilitation by Combining Telerehabilitation With an Exergame in People With Chronic Stroke: Protocol for a Mixed Methods Study. JMIR research protocols. 2020;9:e14629. pmid:32097119
- 51. Blanton S, Clark PC, Lyles RH, Cotsonis G, Jones BD, Reiss A, et al. A web-based carepartner-integrated rehabilitation program for persons with stroke: study protocol for a pilot randomized controlled trial. Pilot and feasibility studies. 2019;5:58. pmid:31057807
- 52. Chaparro D, Daviet JC, Borel B, Kammoun B, Salle JY, Tchalla A, et al. Home-based physical activity incentive and education program in subacute phase of stroke recovery (Ticaa’dom): study protocol for a randomized controlled trial. Trials. 2018;19:1–8.
- 53. Chau JP, Lo SH, Lee VW, Choi KC, Shum EW, Hung ZS, et al. Effectiveness and cost-effectiveness of a virtual multidisciplinary stroke care clinic for community-dwelling stroke survivors and caregivers: a randomised controlled trial protocol. BMJ open. 2019;9:e026500. pmid:31079082
- 54. Chen J, Liu M, Sun D, Jin Y, Wang T, Ren C. Effectiveness and neural mechanisms of home-based telerehabilitation in patients with stroke based on fMRI and DTI: A study protocol for a randomized controlled trial. Medicine. 2018;97:e9605. pmid:29504985
- 55. Gauthier LV, Kane C, Borstad A, Strahl N, Uswatte G, Taub E, et al. Video Game Rehabilitation for Outpatient Stroke (VIGoROUS): protocol for a multi-center comparative effectiveness trial of in-home gamified constraint-induced movement therapy for rehabilitation of chronic upper extremity hemiparesis. BMC neurology. 2017;17:1–8.
- 56. Guillaumier A, McCrabb S, Spratt NJ, Pollack M, Baker AL, Magin P, et al. An online intervention for improving stroke survivors’ health-related quality of life: study protocol for a randomised controlled trial. Trials. 2019;20:1–0.
- 57. Sakakibara BM, Lear SA, Barr SI, Benavente O, Goldsmith CH, Silverberg ND, et al. Development of a chronic disease management program for stroke survivors using intervention mapping: the stroke coach. Archives of physical medicine and rehabilitation. 2017;98:1195–202. pmid:28219685
- 58. Sheehy L, Chapman I, Sviestrup H, Yang C, Bilodeau M, Finestone H. Home-based virtual reality training after stroke: preliminary data of a telerehabilitation feasibility randomized controlled trial. International Journal of Stroke. 2018;13:207.
- 59. Sureshkumar K, Murthy G, Kuper H. Protocol for a randomised controlled trial to evaluate the effectiveness of the ’Care for Stroke’ intervention in India: a smartphone-enabled, carer supported educational intervention for management of disabilities following stroke. BMJ Open. 2018;8:e020098. pmid:29743322
- 60. Hansen CR, Perrild H, Koefoed BG, Zander M. Video consultations as add-on to standard care among patients with type 2 diabetes not responding to standard regimens: a randomized controlled trial. Eur J Endocrinol. 2017;176:727–36. pmid:28325823
- 61.
Australian Physiotherapy Association. Telehealth Guidelines. 2020. https://australian.physio/sites/default/files/APATelehealthGuidelinesCOVID190420FA.pdf.
- 62.
The Physiotherapy Board New Zealand. Telehealth standard. https://www.physioboard.org.nz/standards/physiotherapy-standards/telehealth-standard Accessed on December 12, 2020.
- 63.
Irish Society of Chartered Physiotherapists. Telehealth and physiotherapy. 2020. https://www.iscp.ie/for-the-public/Telehealth. Accessed on December 12, 2020.
- 64.
Chartered Society of Physiotherapy. Digital physiotherapy. 2020. https://www.csp.org.uk/professional-clinical/digital-physiotherapy/telehealth. Accessed on December 12, 2020.
- 65. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. Journal of advanced nursing. 2000;32:1008–15. pmid:11095242
- 66. AHA/ASA Stroke Council Leadership. Temporary Emergency Guidance to US Stroke Centers During the Coronavirus Disease 2019 (COVID-19) Pandemic: On Behalf of the American Heart Association/American Stroke Association Stroke Council Leadership. Stroke. 2020;51:1910–2. pmid:32233972
- 67. Bagot KL, Cadilhac DA, Kim J, Vu M, Savage M, Bolitho L, et al. Transitioning from a single-site pilot project to a state-wide regional telehealth service: The experience from the Victorian Stroke Telemedicine programme. Journal of Telemedicine and Telecare. 2017;23:850–5. pmid:29081268
- 68. Peretti A, Amenta F, Tayebati SK, Nittari G, Mahdi SS. Telerehabilitation: Review of the State-of-the-Art and Areas of Application. JMIR rehabilitation and assistive technologies. 2017;4:e7. pmid:28733271
- 69. Murray E, Hekler EB, Andersson G, Collins LM, Doherty A, Hollis C, et al. Evaluating digital health interventions: key questions and approaches. American Journal of Preventive Medicine. 2016;51:843–851. pmid:27745684
- 70. Prvu Bettger J, Resnik LJ. Telerehabilitation in the age of COVID-19: an opportunity for learning health system research. Physical Therapy. 2020;100:1913–6. pmid:32814976
- 71.
U.S. Centers for Medicare & Medicaid Services, List of telehealth services, 2020. https://www.cms.gov/Medicare/Medicare-General-Information/Telehealth/Telehealth-Codes Accessed on: December 14, 2020.
- 72.
Australian Physiotherapy Association, Announcement of permanent telehealth funding, 2020. https://australian.physio/media/physios-welcome-announcement-permanent-telehealth-funding Accessed on: December 14, 2020.
- 73. Aydın GÖ, Kaya N, Turan N. The role of health literacy in access to online health information. Procedia-Social and Behavioral Sciences. 2015;195:1683–7.
- 74. Bakhtiar M, Elbuluk N, Lipoff JB. The digital divide: how Covid-19’s telemedicine expansion could exacerbate disparities. Journal of the American Academy of Dermatology. 2020;1:e346. pmid:32682890
- 75. Khilnani A, Schulz J, Robinson L. The COVID-19 pandemic: new concerns and connections between eHealth and digital inequalities. Journal of Information, Communication and Ethics in Society. 2020;18:393–403.
- 76. Seah KM. COVID-19: Exposing digital poverty in a pandemic. International Journal of Surgery (London, England). 2020;79:127. pmid:32454251
- 77. Greenhalgh P. How to improve success of technology projects in health and social care. Public Health Research and Practice. 2018;28:e2831815. pmid:30406256
- 78. Kairy D, Lehoux P, Vincent C, Visintin M. A systematic review of clinical outcomes, clinical process, healthcare utilization and costs associated with telerehabilitation. Disability and Rehabilitation. 2009;31:427–447. pmid:18720118
- 79. Caughlin S, Mehta S, Corriveau H, Eng JJ, Eskes G, Kairy D, et al. Implementing Telerehabilitation After Stroke: Lessons Learned from Canadian Trials. Telemed J E Health. 2020 Jun;26(6):710–719. Epub 2019 Sep 9. pmid:31633454.
- 80. Davis F.D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. Management Information Systems Quarterly, 13, 319–340.
- 81. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011 Mar;38(2):65–76. pmid:20957426