https://orcid.org/0000-0001-9784-9729
Abstract
Several measurement tools commonly used in face-to-face neurological rehabilitation have been used in telerehabilitation. However, it is not known whether these tools have adequate measurement properties and clinical utility. This systematic review aims to investigate the measurement properties and the clinical utility of measurement tools used in telerehabilitation in individuals with neurological diseases. A systematic review to investigate the measurement properties and clinical utility of measurement tools used in telerehabilitation in individuals with neurological conditions will be conducted. This systematic review will follow the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) statement. this systematic review protocol was registered in the International Prospective Register of Systematic Reviews (PROSPERO) on 28 May 2021 (registration number: CRD42021257662). Electronic searches will be performed in following databases: Medical Literature Analysis and Retrieval System Online (MEDLINE Ovid), Excerpta Medica Database (Embase Classic + Embase Ovid), Physiotherapy Evidence Database (PEDro), Scientific Electronic Library Online (Scielo), and Literatura Latino-Americana e do Caribe em Ciências da Saúde (LILACS). Two trained independent reviewers will select the studies according to the inclusion criteria, and will also extract the data, evaluate the clinical utility and methodological quality. The relevant data such as design, participants, settings, and mode of administration, measurement properties, and clinical utility will be summarized. Disagreements between reviewers will be resolved by consensus or by the decision of a third independent reviewer. Hand searches of other relevant studies will be employed. The COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) checklist and the clinical utility scale will be used to assess the methodological quality and clinical utility of these tools, respectively. This systematic review will provide information regarding the measurement properties and the clinical utility of the measurement tools used in neurological telerehabilitation. This information will be useful to assist health professionals in choosing adequate measurement tools and planning new research studies.
Citation: Brito SAFd, Scianni AA, Peniche PdC, Faria CDCdM (2022) Measurement properties of outcome measures used in neurological telerehabilitation: A systematic review protocol. PLoS ONE 17(3): e0265841. https://doi.org/10.1371/journal.pone.0265841
Editor: Alastair Smith, University of Plymouth, UNITED KINGDOM
Received: September 10, 2021; Accepted: March 8, 2022; Published: March 21, 2022
Copyright: © 2022 Brito et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The author(s) received no specific funding for this work.
Competing interests: Financial support was provided by the Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG), Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES – Finance Code 001), Conselho Nacional de Desenvolvimento Cientíıfico e Tecnológico (CNPQ) and Pró-reitoria de Pesquisa da Universidade Federal de Minas Gerais (PRPq/UFMG).
Introduction
Neurological conditions have a high prevalence and incidence [1]. These disorders are the leading cause of disabilities and the second biggest cause of death worldwide [1]. From 1990 to 2016, the disabilities and deaths caused by neurological conditions increased approximately 39% and 15%, respectively [1]. It leads to impairments, activity limitations, and participation restrictions [2], as well as negative impacts on the quality of life [3–6]. This makes these individuals commonly in need of rehabilitation services. According to Cieza et al. 2021, neurological conditions were one of the largest determinants of the need for rehabilitation [7]. However, they usually face barriers to access the rehabilitation centers, such as transport problems, lack of a caregiver, high costs, among others [8–10]. In contexts as the COVID-19 pandemic, these barriers may be even greater, as social distancing is recommended to avoid the spread of the virus [11]. Therefore, telerehabilitation has been an innovative alternative to delivery rehabilitation services.
Telerehabilitation can be defined as the remote delivery of rehabilitation services using information and communication technologies, such as telephone, videoconferencing, and sensors [12]. Telerehabilitation sessions can be used to assess, goal‐setting, intervention, education, and monitoring [2]. The use of telerehabilitation has increased over time as technologies become increasingly prevalent and easily accessible [12].
Telerehabilitation has several benefits, such as easier access, can be carried out at the individual’s home, does not require transportation [13], and has a lower cost when compared to face-to-face rehabilitation [14–16]. In recent years, the use of telerehabilitation has increased, mainly due to the development of new computer technologies and more advanced devices that allow long-distance communication [12, 16]. Moreover, during the COVID-19 pandemic, this strategy was even more widespread.
Recent systematic reviews assessed the effectiveness of telerehabilitation in individuals with neurological conditions [17–19]. In these reviews, various outcomes were investigated, and many measurement tools commonly applied in the face-to-face evaluation were used, such as Barthel Index, Berg Balance Scale, Fugl-Meyer Upper Extremity, Action Research Arm Test, and Stroke Impact Scale [17–19]. However, assessing remotely an individual can be challenging [20].
It is important to consider that the results of these measurement tools may be different when comparing telerehabilitation and face-to-face rehabilitation. The measurement tools commonly used in face-to-face rehabilitation may require adaptations for use in telerehabilitation [20]. For example, the Fulg-Meyer Scale [21] commonly used in the face-to-face evaluation of individuals after stroke can be a challenge to remote use [20]. In addition, communication, comprehension, and interaction are different remotely when compared to face-to-face [22]. All these factors can interfere with the measurements provided by the tools. Hence, it is important to establish the measurement properties of the measurement tools, when used in telerehabilitation. The choice of tools with adequate and accurate measurement properties is important to guarantee the quality and reliability of the results, both in research and in clinical practice [23].
As previously discussed, in recent years a wide range of studies have investigated the use of telerehabilitation, consequently, the number of studies investigating the measurement properties of these measurement tools has also increased [17–19, 22]. Systematic reviews are considered the best way to synthesize existing information and provide a comprehensive analysis of the full range of literature on a particular topic [23]. To our knowledge, no systematic review gathered information on the measurement properties and clinical utility of the measurement tools used in telerehabilitation in individuals with neurological conditions. Given the growth of telerehabilitation in research and clinical practice [12, 16] and the challenges of using measurement tools in remote evaluation [20], a systematic review is necessary. Furthermore, this is an important source of information for researchers and professionals to choose measurement tools with adequate measurement properties and clinical utility. Therefore, this systematic review aims to investigate the measurement properties and the clinical utility of measurement tools used in telerehabilitation in individuals with neurological diseases.
Methods
This systematic review protocol was conducted following Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) [24, 25] and the systematic review results will be reported according to Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA 2020 statement) [26].
Study registration
According to PRISMA-P guidelines [24, 25], this systematic review protocol was registered in the International Prospective Register of Systematic Reviews (PROSPERO) on 28 May 2021 (registration number: CRD42021257662).
Eligibility criteria
All full-text papers that aimed to investigate the measurement properties of measurement tools used in telerehabilitation in adults (age ≥18 years old), who had a neurological condition, for example, stroke, Parkinson’s disease, spinal cord injuries, multiple sclerosis, cerebellar ataxia, traumatic brain injuries, and peripheral nervous system diseases, will be included. The searches will not be limited by study design, language, or date of publication.
Research reports, working papers, conference proceedings, conference abstracts, commentaries, letters, dissertations, theses, and editorial papers will be excluded. Systematic reviews and qualitative studies will also be excluded, but their reference lists will be screened for relevant studies.
Search strategy for identification of relevant studies
Electronic search will be carried out for articles indexed on following databases: Medical Literature Analysis and Retrieval System Online (MEDLINE Ovid), Excerpta Medica Database (Embase Classic + Embase Ovid), Physiotherapy Evidence Database (PEDro), Scientific Electronic Library Online (Scielo), and Literatura Latino-Americana e do Caribe em Ciências da Saúde (LILACS). The search strategy was designed according to previous studies and with the assistance of an experienced researcher. The established search strategy for the MEDLINE database will be adapted to suit the other databases. The search strategy (S1 File) is composed of blocks of key terms related to the target population, telerehabilitation, as measurement properties, as follows:
Target population: Individuals with neurological conditions. The search was developed based on a systematic review by Marinho-Buzelli et al. 2015 [27].
Telerehabilitation: The search was developed based on a Cochrane systematic review by Laver et al. 2020 [28].
Measurement properties: The search was developed based on a systematic review by Silva et al. 2014 [29].
Selection of the studies
The selection of the studies will be carried out in three stages. In the first stage, searches will be carried out in the databases. Then, it will be saved and maintained in the Rayyan Systems Inc software [30]. In the second stage, the reviewers will screen the titles and abstracts of the records for eligibility, and the duplicates will be removed. In the third stage, selected full-text articles will be screened for eligibility. The excluded studies and the reason for the exclusion will be recorded. The selection will be performed by two reviewers (SAFB and PCP) independently. Disagreements between reviewers will be resolved by consensus or by the decision of a third independent reviewer (CDCMF). During the selection and screening of the studies, reviewers will be blinded to authors, journals, and outcomes. If additional information would be necessary, the authors of the paper will be contacted. According to the PRISMA 2020 statement [26], a study flow diagram will be created to depict the flow of information through the different stages of this systematic review.
Data extraction
Two reviewers (SAFB and PCP) will independently extract data from the articles using a pre-designed data extraction form. The relevant data extracted from all the included studies will be summarized in tables. Disagreements between reviewers will be resolved by consensus or by the decision of a third independent reviewer (CDCMF).
The tables will contain the following data: study authors; year and country of publication; participants and settings (including data on age, gender, type of neurological condition, and severity of the condition); name, outcomes, and characteristics of the measurement tools; settings and mode of administration (such as telephone, videoconferencing, and sensors); measurement properties (internal consistency, reliability, measurement error, content validity, face validity, construct validity, structural validity, cross-cultural validity, criterion validity, responsiveness); clinical utility; and methodological quality of the study.
Assessment of the methodological quality of the included studies
The methodological quality of the included studies will be assessed using the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) Risk of Bias checklist [31, 32]. In the COSMIN checklist three domains are distinguished (reliability, validity, and responsiveness), and each domain contains one or more measuring properties. The COSMIN checklist is composed of ten boxes (one box for instrument development and nine boxes for the measurement properties). The boxes contain various items, and each item can be scored on a 4‐point rating scale (i.e. very good, adequate, doubtful, inadequate). Standards that are considered not applicable can be skipped. An overall quality score can be obtained by taking the lowest rating for each item in one box (“worst score counts” method) for each measurement property [31–33]. Following the recommendation by Prinsen et al. 2018 [34], the criteria for good measurement properties will also be used [34, 35]. The criteria for good measurement properties present pooled or summarized results per measurement property per measurement tool. Each item can be scored as sufficient (+), insufficient (−), or indeterminate (?) [34, 35]. The assessment will also be carried out by two reviewers independently (SAFB and PCP). Disagreements will be discussed between the two reviewers and, if necessary, a third reviewer (CDCMF) will be consulted.
Assessment of the clinical utility of the identified tools
The clinical utility can be characterized by the ease with which an instrument is incorporated into clinical practice [36, 37]. This can be evaluated by the instrument’s ability to be brief and simple to apply, understand and score [23]. Parameters used to investigate clinical utility include the acceptability (individual friendliness of the instrument, often characterized by the total time to complete the tool and is influenced by the total number of items and the interpretability of the items) and the feasibility (ease of use, such as required specific training, costs, the need of supervision during the completion of the instrument, and time needed to score) [37]. In addition, the clinical utility can be assessed by criteria that may influence the clinicians in using a measurement tool in their practice, such as time to administer, analyze and interpret the measure, cost, need of specialized equipment/training, portability, and accessibility [36]. Studies that have investigated clinical utility using any of these definitions will be included.
Moreover, the clinical utility of all measurement tools will be collected and reported using the scale proposed by Tyson & Connell [36]. This is a 10-point scale that assessed the following criteria:
- Time to administer, analyze and interpret the measure (<10 minutes = 3 points, 10–30 minutes = 2 points, 30–60 minutes = 1 point, >1 hour = 0 points)
- Cost (<£100 = 3 points, £100–£500 = 2 points, £500–£1,000 = 1 point, >£1000 or unknown = 0 point)
- Need of specialized equipment/training (‘No’ = 2 points, ‘Yes, but only simple, easy to use equipment which does not need specialist training’ = 1 point, ‘Yes’ or ‘Unknown’ = 0 points)
- Portability, and accessibility (‘Yes, easily (can go in pocket)’ = 2 points, ‘Yes, in a briefcase or trolley’ = 1 point, ‘No or very difficult’ = 0 points) [36].
The assessment will also be carried out by two reviewers independently (SAFB and PCP).
Data synthesis
A narrative synthesis will be carried out, which will provide texts and tables to synthesize and discuss the data of the studies and methodological characteristics, as previously described in the data extraction section. In addition, texts and tables will be used to summarize the findings regarding the methodological quality [31, 32], quality of measurement properties [33–35], and clinical utility [36, 37].
Discussion
According to our knowledge, this systematic review will be the first one to assess the measurement properties and clinical utility of measurement tools used in telerehabilitation in individuals with neurological conditions. The purpose is to provide a discussion of the strengths and limitations of the different tools used in the evaluation performed during telerehabilitation of individuals with neurological conditions.
Remote delivery health services have been used for a few years. Recently, its use has increased in rehabilitation services, mainly in some specialties such as neurological and cardiac rehabilitation [16]. In the global crisis caused by the COVID-19 pandemic, the rehabilitation services had to adapt quickly to continue offering treatment to the patients [22]. One solution found was telerehabilitation since it allows the rehabilitation of individuals by substituting the traditional face-to-face approach, thus respecting safety rules preconized by health organizations, such as social distancing [38]. Therefore, telerehabilitation has increased exponentially in the last year, both in research and in clinical practice [22].
Investigating the measurement properties of the measurement tools is important to ensure an adequate evaluation of the outcomes [23]. The therapist-patient interaction, characteristics, and settings are different in telerehabilitation when compared to face-to-face rehabilitation [16]. This may reflect different measurement properties compared to those previously investigated in the face-to-face evaluation. Furthermore, functional evaluation has been identified as a challenge to telerehabilitation practice [20, 39]. The identification and use of measurement tools that have appropriate measurement properties and clinical utility may enhance the feasibility and credibility of the evaluation performed remotely, and the comparability of interventions carried out by telerehabilitation. The results of this systematic review will be useful to assist physiotherapists in choosing the measurement tools they will use in telerehabilitation practice. Moreover, it can direct the definition of future research goals and the planning of new research studies.
This review employs a systematic, clear, and replicable inclusion and exclusion criteria and search strategy, as well as the approach regarding the searching, screening, and extracting data. The methods and instruments used in this study are recommended and validated. The involvement of two reviewers in all stages from the selection to the data extraction phase, as well as the assessment of the methodological quality of the included studies, will enhance the methodological rigor and credibility of the results found. The results from this systematic review will be spread by scientific peer-review publications and presentations at conferences and scientific events.
Supporting information
S1 Checklist. PRISMA-P (Preferred Reporting Items for Systematic review and Meta-Analysis Protocols) 2015 checklist: Recommended items to address in a systematic review protocol*.
https://doi.org/10.1371/journal.pone.0265841.s001
(DOC)
References
- 1. Feigin VL, Nichols E, Alam T, Bannick MS, Beghi E, Blake N, et al. Global, regional, and national burden of neurological disorders, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol. 2019; 18(5):459–480. pmid:30879893
- 2.
World Health Organization. Neurological disorders: public health challenges. World Health Organization. 2006.
- 3. De Wit L, Theuns P, Dejaeger E, Devos S, Gantenbein AR, Kerckhofs E, et al. Long-term impact of stroke on patients’ health-related quality of life. Disabil Rehabil. 2017; 39(14):1435–1440. pmid:27385479
- 4. Kuopio AM, Marttila RJ, Helenius H, Toivonen M, Rinne UK. The quality of life in Parkinson’s disease. Movement disorders: official journal of the Movement Disorder Society. 2000; 15(2):216–223. pmid:10752569
- 5. Boakye M, Leigh BC, Skelly AC. Quality of life in persons with spinal cord injury: comparisons with other populations. J Neurosurg Spine. 2012; 17(1):29–37. pmid:22985368
- 6. Mccabe MP, Mckern S. Quality of life and multiple sclerosis: Comparison between people with multiple sclerosis and people from the general population. Journal of Clinical Psychology in Medical Settings. 2002; 9(4):287–295.
- 7. Cieza A, Causey K, Kamenov K, Hanson SW, Chatterji S, Vos T, et al. Global estimates of the need for rehabilitation based on the Global Burden of Disease study 2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet. 2021; 396(10267):2006–2017. pmid:33275908
- 8. Domingos J, Radder D, Riggare S, Godinho C, Dean J, Graziano M, et al. Implementation of a Community-Based Exercise Program for Parkinson Patients: Using Boxing as an Example. J Parkinsons Dis. 2019; 9(3):615–623. pmid:31282426
- 9. Scelza WM, Kalpakjian CZ, Zemper ED, Tate DG. Perceived barriers to exercise in people with spinal cord injury. Am J Phys Med Rehabil. 2005; 84(8):576–83. pmid:16034226
- 10. Rimmer JH, Wang E, Smith D. Barriers associated with exercise and community access for individuals with stroke. J Rehabil Res Dev. 2008; 45(2):315–22. pmid:18566948
- 11. Palmieri L, Palmer K, Noce CL, Meli P, Giuliano M, Floridia M, et al. Differences in the clinical characteristics of COVID-19 patients who died in hospital during different phases of the pandemic: national data from Italy. Aging Clin Exp Res. 2021; 33(1):193–199. pmid:33345291
- 12. Brennan DM, Mawson S, Brownsell S. Telerehabilitation: enabling the remote delivery of healthcare, rehabilitation, and self management. Stud Health Technol Inform. 2009; 145:231–48. pmid:19592797
- 13. Hjelm NM. Benefits and drawbacks of telemedicine. J Telemed Telecare. 2005; 11(2):60–70. pmid:15829049
- 14. Tousignant M, Moffet H, Nadeau S, Mérette C, Boissy P, et al. Cost analysis of in-home telerehabilitation for post-knee arthroplasty. J Med Internet Res. 2015; 17(3):e83. pmid:25840501
- 15. Hwang R, Morris NR, Mandrusiak A, Bruning J, Peters R, Korczyk D, et al. Cost-Utility Analysis of Home-Based Telerehabilitation Compared With Centre-Based Rehabilitation in Patients With Heart Failure. Heart Lung Circ. 2019; 28(12):1795–1803. pmid:30528811
- 16. Peretti A, Amenta F, Tayebati SK, Nittari G, Mahdi SS. Telerehabilitation: Review of the State-of-the-Art and Areas of Application. JMIR Rehabil Assist Technol. 2017; 4(2):e7. pmid:28733271
- 17. Tchero H, Tabue-Teguo M, Lannuzel A, Rusch E. Telerehabilitation for stroke survivors: systematic review and meta-analysis. Journal of medical Internet research. 2018; 20(10):e10867. pmid:30368437
- 18. Ownsworth T, Arnautovska U, Beadle E, Shum DH, Moyle W. Efficacy of Telerehabilitation for Adults With Traumatic Brain Injury: A Systematic Review. J Head Trauma Rehabil. 2018; 33(4):E33–E46. pmid:29084100
- 19. Di Tella S, Pagliari C, Blasi V, Mendozzi L, Rovaris M, Baglio F. Integrated telerehabilitation approach in multiple sclerosis: A systematic review and meta-analysis. J Telemed Telecare. 2020; 26(7–8):385–399. pmid:31132909
- 20. Chirra M, Marsili L, Wattley L, Sokol LL, Keeling E, Maule S, et al. Telemedicine in Neurological Disorders: Opportunities and Challenges. Telemed J E Health. 2019; 25(7):541–550. pmid:30136898
- 21. Fugl-Meyer AR, Jääskö L, Leyman I, Olsson S, Steglind S. The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance. Scand J Rehabil Med. 1975; 7(1):13–31. pmid:1135616
- 22. Annaswamy TM, Verduzco-Gutierrez M, Frieden L. Telemedicine barriers and challenges for persons with disabilities: COVID-19 and beyond. Disabil Health J. 2020; 13(4):100973. pmid:32703737
- 23.
Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. Pearson/Prentice Hall; 2009.
- 24. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015; 4(1). pmid:25554246
- 25. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. Bmj. 2015; 349. pmid:25555855
- 26. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. MetaArXiv. 2020.
- 27. Marinho-Buzelli AR, Bonnyman AM, Verrier MC. The effects of aquatic therapy on mobility of individuals with neurological diseases: a systematic review. Clin Rehabil. 2015; 29(8):741–51. pmid:25394397
- 28. Laver KE, Adey‐Wakeling Z, Crotty M, Lannin NA, George S, Sherrington C. Telerehabilitation services for stroke. Cochrane Database Syst Rev. 2020; 1:CD010255. pmid:32002991
- 29. Silva PF, Quintino LF, Franco J, Faria CD. Measurement properties and feasibility of clinical tests to assess sit-to-stand/stand-to-sit tasks in subjects with neurological disease: a systematic review. Braz J Phys Ther. 2014; 18(2):99–110. pmid:24839043
- 30. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Systematic reviews. 2016; 5(1):1–10. pmid:27919275
- 31. Mokkink LB, De Vet HC, Prinsen CA, Patrick DL, Alonso J, Bouter LM, et al. COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures. Quality of Life Research. 2018; 27(5):1171–1179. pmid:29260445
- 32. Mokkink L. COSMIN risk of bias checklist: Amsterdam Public Heal. Res. Inst. 2018. pmid:29260445
- 33. Mokkink LB, Boers M, van der Vleuten CPM, Bouter LM, Alonso J, Patrick DL, et al. COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study. BMC medical research methodology. 2020; 20(1):1–13. pmid:33267819
- 34. Prinsen CA, Mokkink LB, Bouter LM, Alonso J, Patrick DL, De Vet HC, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018; 27(5):1147–1157. pmid:29435801
- 35. Terwee CB, Mokkink LB, Knol DL, Ostelo RW, Bouter LM, de Vet HC. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012; 21(4);651–7. pmid:21732199
- 36. Tyson S; Connell L. The psychometric properties and clinical utility of measures of walking and mobility in neurological conditions: a systematic review. Clin Rehabil. 2009; 23(11):1018–33. pmid:19786420
- 37. Valier AR, Lam KC. Beyond the basics of clinical outcomes assessment: selecting appropriate patient-rated outcomes instruments for patient care. Athletic Training Education Journal. 2015; 10(1), 91–100.
- 38.
World Health Organization. Transmission of SARS-CoV-2: implications for infection prevention precautions: scientific brief. World Health Organization. 2020.
- 39. Appleby E, Gill ST, Hayes LK, Walker TL, Walsh M, Kumar S. Effectiveness of telerehabilitation in the management of adults with stroke: A systematic review. PloS one. 2019;14(11):e0225150. pmid:31714924