Valid and multidisciplinary assessment of a stroke patient's ability to perform activities of daily living is very important to define individual goals and to plan targeted rehabilitation. Until today, there is no observation scale that relies on International Classification of Functioning, Disability and Health (ICF). The aim of the present study was to develop and evaluate the reliability and validity of a new multidisciplinary observation scale for stroke patients, shortly called LIMOS, which is based on ICF.
In a first phase, LIMOS was defined, using a Delphi approach, by an expert panel and a pilot testing was conducted in a small group of stroke patients (n =10) to investigate feasibility and practicability. In a second phase, LIMOS was assessed for its reliability (internal consistency and test-retest reliability) and validity in a large cohort of stroke patients (n = 102). For convergent validity, the correlation between total scores of the LIMOS and the Functional Independence Measure (FIM) was assessed.
LIMOS consisted of seven ICF chapters incorporating 45 domains. A high internal consistency (=0.98) of LIMOS was found. Furthermore, good test-retest reliability at item and subscale level was found. Principal component analysis revealed that among the seven ICF chapters, four components could be found: (1) interpersonal activities, mobility and self-care, (2) communication, (3) knowledge and general tasks, and (4) domestic life. Significant associations were found between LIMOS and the FIM indicating good convergent validity.
Citation: Ottiger B, Vanbellingen T, Gabriel C, Huberle E, Koenig-Bruhin M, Plugshaupt T, et al. (2015) Validation of the New Lucerne ICF Based Multidisciplinary Observation Scale (LIMOS) for Stroke Patients. PLoS ONE 10(6): e0130925. https://doi.org/10.1371/journal.pone.0130925
Editor: Johannes Boltze, Fraunhofer Institute for Cell Therapy and Immunology, GERMANY
Received: February 23, 2015; Accepted: May 27, 2015; Published: June 25, 2015
Copyright: © 2015 Ottiger et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This work was supported by the Swiss National Science Foundation (Grant Number 320030_140696).
Competing interests: The authors have declared that no competing interests exist.
An estimated 50 million stroke survivors worldwide currently cope with significant physical, cognitive and emotional deficits. Up to 74% of these survivors still require some assistance or fully depend on the aid of a caregiver when they perform activities of daily living (ADL) such as grooming or walking outside [1– 3]. Several neurological dysfunctions (e.g. motor, sensory, visual) and neuropsychological deficits (memory, attention, language) may explain why stroke patients experience disability in ADL, in severe cases even years after discharge from the rehabilitation center .
To optimize functional outcome of a stroke patient, a valid detection of a patient’s ability to perform ADL is unequivocal at the start as well as at the end of rehabilitation procedures. This enables clinicians to set measurable treatment goals, to make appropriate discharge arrangements, or to anticipate the need for community support [3, 5]. To measure disability, clinicians and researchers often use basic ADL measurements, such as the Barthel-Index, the extended Barthel Index, modified Ranking Scale (see for an overview Weimar et al. 2002, ) or the Functional Independent Measurement (FIM) [5, 7, 8]. Of all these measurements the FIM, is worldwide one of the most often used measurements. It is a clinician-reported valid measure of disability that has been widely applied for evaluating recovery from stroke. Although FIM has some drawbacks (e.g. ceiling and floor effects; focus on physical domains) and was not primary meant to be comprehensive, it is often used as a gold standard tool to measure ADL [5, 7, 8].
The International Classification of Functioning, Disability and Health (ICF) framework set by the World Health Organization (WHO) was developed to optimize a common language among clinicians  and has become a standard in neurorehabilitation. Based on this new classification, ICF core-sets were established for stroke [10, 11]. This facilitated the linking of ICF domains with existing standardized measurements [3, 8, 12, 13]. Recently, new measurements incorporating ICF domains were developed, in the form of self–reported questionnaires [12, 14], or mono disciplinary observation tools for nurses , physiotherapists  and occupational therapists . The disadvantage of self–reported questionnaires is, however, that they strongly depend on preserved cognitive abilities (i.e. insight) of a stroke patient, possibly biasing the reliability of their answers. Mono disciplinary observation tools, on the other hand, have the disadvantage that they do not reflect the whole condition of a stroke patient, within a multi-disciplinary rehabilitation setting. A valid and reliable comprehensive multidisciplinary observation tool based on ICF for stroke patients is still missing.
The aim of the present study was to develop and evaluate the reliability and validity of a new comprehensive multidisciplinary ICF based observation scale, shortly called LIMOS, in a large cohort of stroke patients. We predicted good test-retest reliability at domain and subscale level of LIMOS. We expected a good internal consistency of the total scale and a homogeneous structure of the different subscales. Furthermore, we hypothesized that the LIMOS will be associated with the FIM, indicating good convergent validity of the scale.
Materials and Methods
The study was approved by the ethical committee of the state of Lucerne and was consistent with the Declaration of Helsinki, 1975. All patients gave written informed consent prior to participation.
Phase 1: Development of the LIMOS
Development of the main construct of the LIMOS.
Experts from the Neurology and Neurorehabilitation Center, Luzerner Kantonsspital, were defined for the development of LIMOS (2 neurologists, 3 nurses, 1 occupational therapist, 1 speech therapist, 3 physiotherapists, and 1 neuropsychologist). This center has a long tradition in using and implementing the ICF since 2002 and already developed an internal simplified ICF manual . Using a Delphi approach, in a consensus round table the chapters (learning and applying knowledge, general tasks and demands, communication, mobility, self-care, domestic life, interpersonal interactions and relationships, major life areas, community, social and civic life) and their incorporating domains (i.e. focusing attention (d160), solving simple problems (d1750), lifting and carrying objects (d430), caring for body parts (d520) etc.) were evaluated.
LIMOS was then presented and discussed within a multidisciplinary workshop. All opinions and remarks were reviewed by the expert panel and further adjustments were done until full agreement was obtained on 9 chapters including 51 domains for the LIMOS.
A new scoring system for the LIMOS was then evaluated. According to the ICF, limitations or restriction in domains are classified as “none”, “slight”, “moderate”, “severe” or “complete” . Regarding the discharge planning after inpatient rehabilitation, it is essential to know how patients perform ADL, i.e. whether they are independent or whether they need further assistance. Therefore, the grade of assistance is a crucial determinant in defining whether someone may be able to return at home or not. Consequently, to better rate limitation or restriction in the clinical setting, we adapted the rating of the ICF domains. A 5 point scale for the LIMOS was thus defined: 1 = patient is not able to fulfil a task or need assistance up to 75% (corresponding to “complete”); 2 = patient is able to fulfil tasks with assistance of 25% to 75% (corresponding to “severe”); 3 = patients is able to fulfil tasks with assistance less than 25% or under supervision (corresponding to “moderate”); 4 = patient is able to fulfil tasks independently but needs more time and/or with auxiliary materials, aids (corresponding to “slight”); 5 = patient is able to fulfil tasks independently (corresponding to “none”).
A pilot test of the LIMOS in patients with stroke.
A pilot test was conducted in 10 stroke patients to examine the feasibility and practicability of the LIMOS. The responsible neurologists, nurses, physiotherapists, occupational-therapists and speech-therapists of these patients rated their assigned domains of the LIMOS, and these within 72 hours after patient’s admission, and within the last 72 hours before discharge. These members of this pilot phase were not the same as the members of the expert panel. The LIMOS was easy and short to administer, taking only five to ten minutes.
After the pilot test, 7 of the 9 predefined chapters were determined, within an expert panel meeting (members of the development phase), these chapters being: learning and applying knowledge, general tasks and demands, communication, mobility, self-care, domestic life, interpersonal interactions and relationships). The chapters ‘community, social and civic life’ and ‘major life’ were excluded by consensus, since domains such as recreation, religion and political life, within this chapters are not main priority in an inpatient rehabilitation setting.
Based on the review of the expert panel review and the pilot testing, the final construct of the LIMOS was defined, which included 7 chapters incorporating 45 domains (see S1 Table). Based on the 5-point scale (1–5) total score ranges from 45 (lowest) to 225 (highest).
Phase 2: Examination of the reliability and validity of the LIMOS
In a second phase the reliability and validity of the LIMOS was assessed in a new large group of stroke patients.
One hundred and two in-patients with stroke (33 women, aged between 41–89 years, mean = 69.3, SD = 11.9, time post stroke between 1–69 days, mean = 12.9, SD = 9.7) were prospectively included. Sixty-five patients had an ischemic (A. cerebri media = 43, A. cerebri posterior = 11, A. cerebri anterior = 5) and 37 patients had a hemorrhagic stroke (A. cerebri media = 10, A. cerebri posterior = 1, A. cerebri anterior = 1, others = 25). All patients were admitted to the Neurology and Neurorehabilitation Center, Luzerner Kantonsspital (LUKS), from January to October 2014.
Each stroke patient was assessed with LIMOS and the FIM within 72 hours from admission. The FIM is a standardized assessment for ADL, which includes 18 items rated on a 7-point scale: 1 = total assistance; 2 = maximal assistance; 3 = moderate assistance; 4 = minimal contact assistance; 5 = supervision or set-up; 6 = modified independence; and 7 = complete independence . Total score ranges from 18 (lowest) to 126 (highest). In rehabilitation, the subtotal-summed scores of motor and cognitive subscales (FIM-motor and FIM-cognition) are commonly used to quantify functional independence. The FIM was rated by a multi-discliplinary team.
Descriptive analyses were applied to patients’ demographic variables and behavioral performances (LIMOS, FIM). Internal consistency of the total LIMOS was examined using Cronbach’s alpha. A value above 0.70 is an indicator of a satisfactory homogeneity of items within the total scale. For clinical applications, values above 0.90 are recommended .Test-retest reliability of item scores was examined by computing the Kappa of all item scores in a subgroup of 30 patients, which were assessed twice by the same rater (mean delay between test and retest was 2 days; the range was one week). In another group of 15 patients, inter-rater reliability of item scores was assessed by two independent raters. According to Landis and Koch  a kappa value between 0.21–0.40 indicates “fair” agreement, between 0.41 and 0.60 “moderate” agreement, between 0.61 and 0.80 “substantial”, and above 0.80 “almost perfect”. Agreement between subscale scores was examined using intra-class correlations (ICC). ICC values should be at least 0.75 . Construct validity was analysed by examining the dimensionality of LIMOS by using a principal component analysis. Convergent validity was assessed by using Pearson’s correlation analyses to assess the association between the LIMOS total scale and the FIM total scale. In addition, associations between subscales of the LIMOS and the subscales FIM mobility and FIM cognition were tested. Scales measuring similar concepts should show correlations, r > 0.60. Furthermore, an association between the LIMOS subscale and the Barthel Index, which was indirectly derived from the motor items of the FIM , was tested.
Level of significance was set at p = 0.05 (two-tailed). All values are expressed as mean ± standard deviation (SD). Statistical analyses were performed using PASW for Windows (version 22.0; SPSS, Inc. Chicago, IL).
The total scores of the LIMOS ranged between 45 and 221 (mean = 144.8, SD = 45.3), and for the total FIM from 18 to 126 (mean = 86.9, SD = 31.4).
An excellent internal consistency of 0.98 for the total LIMOS scale (n = 45 items) was found. Regarding the test-retest reliability at item level, the majority of items had moderate (n = 43) to almost perfect agreement (range of kappa 0.41–0.84). Two items (n = 2) showed fair agreement (0.32–0.37) only, i.e. washing oneself (d510) and coping night (d598). Intra-class correlations for all the subscales were > 0.75 (range of ICC values 0.76–0.0.95). Inter-rater reliability analyses at item level revealed that most items had moderate (n = 33) to almost perfect agreement (range of kappa 0.41–0.92). Twelve items showed fair agreement.
The principal components analysis resulted in 4 components with an eigenvalue above 1.0. A strong first component had an eigenvalue of 28.52 and explained, 55.92% of the variance. A second component had an eigenvalue of 4.60 and explained 9.02% of the variance. A third component had an eigenvalue of 2.90 and explained 5.69% of the variance. A fourth component had an eigenvalue of 1.94 explaining in total a further 3.80% of the variance. Communalities were between 0.53 and 0.96, also indicating substantial common variance. After varimax rotation the 4 components reflected interpersonal activities, mobility and self-care as a first component, communication as a second, knowledge and general tasks as a third, domestic life as a fourth. Internal consistency of these 4 components ranged from 0.95 to 0.97.
A strong significant association between the total LIMOS scale and the total FIM scale was found indicating excellent convergent validity (r = 0.89, p < 0.0001). Furthermore strong significant associations were found between the subscales LIMOS mobility and FIM mobility (r = 0.90, p < 0.0001) and the subscales LIMOS knowledge and FIM cognition (r = 0.81, p < 0.0001). Weaker associations between the other subscales of the LIMOS (self-care, general tasks, domestic life) and the subscales of the FIM were found and ranged between r = 0.36–0.79, p < 0001). In addition a significant association (r = 0.92, p < 0.0001) between the subscale LIMOS mobility and the Barthel Index, which was indirectly derived from the motor items of the FIM , was found.
The present study shows that the new Lucerne ICF based Multidisciplinary Observation Scale (LIMOS), which is based on ICF, is a valid and reliable multidisciplinary observation tool for stroke patients in a rehabilitation setting. The construct of the LIMOS was found to be very homogeneous, including four major components. Furthermore, high test-retest reliability at subscale level of LIMOS was found. At domain level, the majority of domains showed moderate (n = 43) to almost perfect agreement. The remaining two domains (i.e. washing oneself (d510) and coping night (d598)) showed fair agreement.
We used the FIM as a criterion to examine the convergent validity of the LIMOS. Our results showed a high degree of correlation between the two scales as well as their subscales, indicating that both instruments measure similar constructs. These results confirm our hypothesis and support the convergent validity of LIMOS in stroke patients. Combining the results of a high uni-dimensionality and convergent validity of the LIMOS, the construct validity of the LIMOS is highly supported. Compared with the FIM, mobility and cognitive aspects in ADL can be assessed in a more detailed way by LIMOS. Furthermore, other aspects of daily functioning are also assessed such as self-care, general tasks and domestic life, which may explain the weaker associations found between these subscales and the subscales of the FIM. Consequently, the LIMOS is more comprehensive than the FIM. Having said that, the LIMOS is still easy and short to administer, as each discipline rates their own subpart within the whole scale, requiring only five to ten minutes per discipline. The short administration time of the scale is thus another important advantage of the LIMOS in busy rehabilitation centers. The current trend to shorten the length of hospital stay in rehabilitation settings, and the increasing demand for efficiency in the continuum of stroke care, imply that knowledge about the outcome of ADL is crucial to optimize stroke management. The LIMOS provides a realistic goal-setting and enables early discharge planning.
When developing new outcome scales, it is recommended that these should incorporate the ICF framework, therefore increasing their relevance in clinical practice . Since the LIMOS is based on the ICF, it provides a common understanding and language of functioning, disability, and health used by physicians, nurses, therapists, and other health professionals [4, 25]. The fact that the LIMOS differentiates from basic to complex ADL’s, possible floor and ceiling effects—which have been described for the FIM [5, 26]—can be avoided. Every patient, mildly or severely affected, can be observed and classified with LIMOS. To assess subjective individual goals of a patient, self-reported ICF questionnaires are still needed, and for specific sensory-motor and cognitive dysfunctions, standardized assessments should still be used. Recovery after stroke can be the result of both substitution and restitution within the defined levels of the ICF classification . LIMOS describes the possibilities of a patient in performing activities, irrespective whether recovery involves restitution (e.g. without hand splint) or substitution (e.g. with hand splint). It is therefore essential that in the clinical setting, therapists interpret the LIMOS in the context of compensatory aids.
A limitation of the present study is that LIMOS is not yet validated for other neurological diseases. Regarding test-retest two items showed fair test-retest reliability (i.e. washing oneself (d510) and coping night (d598)). Furthermore, twelve items showed fair inter-rater reliability. All these items we still kept due to their high content validity.
In conclusion, LIMOS is a new valid observation scale based on the ICF, which can be used by a multidisciplinary team working in a neurorehabilitation setting. The LIMOS is a useful and comprehensive tool to assess patients’ ADL functions, and to identify patients’ difficulties in performing ADL tasks. It can be used as a standard tool to plan intervention strategies and to assess functional outcome over a longer period of recovery after stroke.
S1 Table. The LIMOS.
The detailed chapters and domains of the LIMOS are shown. Scoring ranges from 1–5. (1 = patient is not able to fulfil a task or needs assistance up to 75%; 2 = patient is able to fulfil tasks with assistance of 25% to 75%; 3 = patients is able to fulfil tasks with assistance less than 25% or under supervision; 4 = patient is able to fulfil tasks independently but needs more time and/or with auxiliary materials, aids; 5 = patient is able to fulfil tasks independently).
We are very grateful for the cooperation of all nurses, physiotherapists, occupational therapists, speech-therapists, neuropsychologists and physicians in the Neurology and Neurorehabilitation Center Lucerne involved in the development of LIMOS and data collection. A booklet will be published on our website (https://www.luks.ch/standorte/luzern/kliniken/zentrum-fuer-neurologie-und-neurorehabilitation-znn/forschung.html) with examples of assessments and interpretations of the LIMOS.
Conceived and designed the experiments: BO TV SB TN. Performed the experiments: BO TV CG TP. Analyzed the data: BO TV. Contributed reagents/materials/analysis tools: BO TV CG EH MK TP SB TN. Wrote the paper: BO TV CG EH MK TP SB TN.
- 1. Evans A, Harraf F, Donaldson N, Kalra L. Randomized controlled study of stroke unit care versus stroke team care in different stroke subtypes. Stroke 2002, 33:449–455. pmid:11823651
- 2. Miller EL, Murray L, Richards L, Zorowitz R, Bakas T, Clark P & et al. Comprehensive Overview of Nursing and Interdisciplinary Rehabilitation Care of Stroke Patient. Stroke 2010, 41:2402–2448. pmid:20813995
- 3. Müller M, Grill E, Stier-Jarmer M, Strobl R, Gutenbrunner C, Fialka-Moser V et al. Validation of the comprehensive ICF core sets for patients receiving rehabilitation interventions in the acute care setting. J Rehabil Med. 2011, 43:92–101. pmid:21234510
- 4. Frank M, Conzelmann M, Engelter S. Prediction of Discharge Destination after Neurological Rehabilitation in Stroke Patients. European Neurology 2010, 63:227–233. pmid:20215754
- 5. Kwon S, Hartzema AG, Duncan PW, Min-Lai S. Disability Measures in Stroke: Relationship Among the Barthel Index, the Functional Independence Measure, and the Modified Rankin Scale. Stroke 2004, 35:918–923. pmid:14976324
- 6. Weimar C, Kurth T, Kraywinkel K, Wagner M, Busse O, Haberl RL,et. al.; Assessment of functioning and disability after ischemic stroke. Stroke 2002, 33(8):2053–9 pmid:12154262
- 7. Grill E, Scheuringer M, Melvin J. Validation of International Classification of Functioning, Disability, and Health (ICF) Core Sets for Early Postacute Rehabilitation Facilities. Am. J. Phys. Med. Rehabil 2006, 85:640–649. pmid:16865018
- 8. Lin JH, Lo S, Chang Y, Su C, Liu C. Validation of comprehensive Assessment of Activities of Daily Living in Stroke Survivors. J Med Sci 2004, 20:287–293. pmid:15253470
- 9. WHO, World Health Organization. International Classification of Functioning, Disability and Health: ICF. Geneva. 2001
- 10. Cieza A, Ewert T, Üstün B, Chatterji S, Kostanjsek N, Stucki G. Development of ICF Core Sets for patients with chronic conditions. J Rehab Med 2004, 36:9–11.
- 11. Stucki G, Grimby G. Foreword: Applying the ICF in medicine. J Rehab Med 2004, 36:5–6.
- 12. Post MWM, de Witte L, Reichrath E, Verdonschot M, Wijlhuizen G, Perenboom R. Development and Validation of Impact-S, an ICF-based questionnaire to measure activities and participation. J Rehabil Med 2008, 40:620–627. pmid:19020695
- 13. Noonan V, Kopec J, Noreau L, Singer J, Chan A, Mâsse L et al. Comparing the content of participation instruments using the Internation Classificiation of Functioning, Disability and Health. BioMed Central Ltd 2009, 7:43–93.
- 14. Farin E, Fleitz A, Frey C. Psychometric Properties of an International Classification of Functioning, Disability and Health (ICF)-oriented, adaptive questionnaire for the assessment of mobility, self-care and domestic life. J Rehabil Med 2007, 39:537–546. pmid:17724553
- 15. Persoon A, Joosten-Weyn Banningh L, van de Vrie W, Olde Rikkert M, van Achterberg. Development of the Nurses' Observation Scale for Cognitive Abilities (NOSCA). ISRN Nursing (895082). 2011
- 16. Huber E, Tobler A, Gloor-Juzi T, Grill E, Gubler-Gut B. The ICF as a way to specify goals and to assess the outcome of physiotherapeutic interventions in the acute hospital. J Rehabil Med. 2011, 43:1650–1977.
- 17. Glässel A, Kirchberger I, Linseisen E, Stamm T, Cieza A, Stucki G. Content validation of the Internation Classification of Functioning, Disability and Health (ICF) Core Set for stroke: The perspective of occupational therapists. Canadian Journal of Occupational Therapy 2010, 77:289–302. pmid:21268511
- 18. Rentsch HP, Bucher P, Dommen Nyffeler I, Wolf C, Hefti H, Fluri E et al. The implementation of the 'International Classificiation of Functioning, Disability and Health' (ICF) in daily practice of neurorehabilitation: an interdisciplinary project at the Kantonsspital of Lucerne, Switzerland. Disabil Rehabil 2003, 411–21. pmid:12745951
- 19. Granger C, Hamilton B, Keith R, Zielezny M, Sherwin F. Advances in functional assessment for medical rehabilitation. 1986, TGR1:59–74.
- 20. Bland JM, Altman DG Cronbach's alpha. BMJ: Britsh Medical Journal 1997, 314:(7080):572.
- 21. Landis JR, Koch GG. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics 1977, 33(2):363–74. pmid:884196
- 22. Fleiss JL. Reliability of measurement. In: The Design and Analysis of Clinical Experiments. New York: John Wiley and Sons, 1986, 1–31
- 23. Nyein K, McMichael L, Turner-Stokes L. Can a Barthel score be derived from the FIM? Clin Rehabil. 1999, 13(1):56–63. pmid:10327098
- 24. Baker K, Cano SJ, Playford ED. Outcome Measurement in Stroke: A Scale Selection Strategy. Stroke 2011, 42:1787–1794. pmid:21566236
- 25. Garin O, Ayuso-Mateos J, Almansa J, Nieto M, Chatterji S, Vilagut G et al. Validation of the "World Health Organization Disability Assessment Schedule, WHODAS-2" in patient with chronic diseases. Health Qual. Life Outcomes 2010, 19:8–51.
- 26. Desrosiers J, Rochette A, Noreau L, Bravo G, Hébert R, Boutin C. Comparison of two functional independence scales with a participation measure in post-stroke rehabilitation. Arch Gerontol Geriatr. 2003, 37 (2):157–72. pmid:12888229
- 27. Kwakkel G, Buma FE, Selzer ME. Understanding the mechanisms underlying recovery after stroke. In Textbook of neural repair and Rehabilitation 2e, eds. Selzer Michael E., Clarke Stephanie, Cohen Leonardo G., Kwakkel Gert, and Miller Robert H.; Cambridge University Press, 2014