Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Psychometric validation of the Italian Rehabilitation Complexity Scale-Extended version 13

  • Francesca Roda ,

    Roles Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    francesca.roda@unipr.it

    Affiliation Department of Medicine and Surgery, Unit of Neuroscience, University of Parma, Parma, Italy

  • Maurizio Agosti,

    Roles Data curation, Formal analysis, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Rehabilitation Medicine Service, Rehabilitation Geriatrics Department of the NHS-University Hospital of Parma, Parma, Italy

  • Andrea Merlo,

    Roles Data curation, Methodology, Software, Writing – review & editing

    Affiliation Motion Analysis Laboratory, Department of Rehabilitation, "S. Sebastiano" Hospital of Correggio, NHS Local Agency of Reggio Emilia, Reggio Emilia, Italy

  • Maurizio Maini,

    Roles Conceptualization, Investigation, Resources, Writing – review & editing

    Affiliation “San Giacomo” Hospital, Ponte dell’Olio, Piacenza, Italy

  • Francesco Lombardi,

    Roles Conceptualization, Investigation, Resources, Writing – review & editing

    Affiliation Neurorehabilitation Service, "S. Sebastiano" Hospital of Correggio, NHS Local Agency of Reggio Emilia, Reggio Emilia, Italy

  • Claudio Tedeschi,

    Roles Conceptualization, Investigation, Resources, Writing – review & editing

    Affiliation Physical medicine and Rehabilitation Unit – Neuromotor Department, IRCCS "Arcispedale Santa Maria Nuova" of Reggio Emilia, Reggio Emilia, Italy

  • Maria Grazia Benedetti,

    Roles Conceptualization, Investigation, Resources, Visualization

    Affiliation Physical Medicine and Rehabilitation Unit, “Rizzoli” Orthopaedics Hospital, Bologna, Italy

  • Nino Basaglia,

    Roles Conceptualization, Investigation, Resources, Writing – review & editing

    Affiliation Department of Neuroscience and Rehabilitation, University Hospital of Ferrara, Ferrara, Italy

  • Mara Contini,

    Roles Conceptualization, Investigation, Resources, Writing – review & editing

    Affiliation Extensive Orthopaedic Rehabilitation Unit, Department of Medicine, Borgo Val Di Taro Hospital, NHS Local Agency of Parma, Parma, Italy

  • Domenico Nicolotti,

    Roles Conceptualization, Investigation, Resources, Writing – review & editing

    Affiliation Intensive Rehabilitation Medicine Spinal Unit, Emergency Medicine Department, Villanova d’Arda Hospital, NHS Local Agency of Piacenza, Piacenza, Italy

  • Rodolfo Brianti,

    Roles Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Rehabilitation Medicine Service, Rehabilitation Geriatrics Department of the NHS-University Hospital of Parma, Parma, Italy

  • on behalf of GRECo

    Membership list can be found in the Acknowledgments section

Psychometric validation of the Italian Rehabilitation Complexity Scale-Extended version 13

  • Francesca Roda, 
  • Maurizio Agosti, 
  • Andrea Merlo, 
  • Maurizio Maini, 
  • Francesco Lombardi, 
  • Claudio Tedeschi, 
  • Maria Grazia Benedetti, 
  • Nino Basaglia, 
  • Mara Contini, 
  • Domenico Nicolotti
PLOS
x

Abstract

In Italy, at present, a well-known problem is inhomogeneous provision of rehabilitative services, as stressed by MoH, requiring appropriate criteria and parameters to plan rehabilitation actions. According to the Italian National Rehabilitation Plan, Comorbidity, Disability and Clinical Complexity should be assessed to define the patient’s real needs. However, to date, clinical complexity is still difficult to measure with shared and validated tools. The study aims to psychometrically validate the Italian Rehabilitation Complexity Scale-Extended v13 (RCS-E v13), in order to meet the guidelines requirements. An observational multicentre prospective cohort study, involving 8 intensive rehabilitation facilities of the Emilia-Romagna Region and 1712 in-patients, [823 male (48%) and 889 female (52%), mean age 68.34 years (95% CI 67.69–69.00 years)] showing neurological, orthopaedic and cardiological problems, was carried out. The construct and concurrent validity of the RCS-E v13 was confirmed through its correlation to Barthel Index (disability) and Cumulative Illness Rating Scale (comorbidity) and appropriate admission criteria (not yet published), respectively. Furthermore, the factor analysis indicated two different components (“Basic Care or Risk—Equipment” and “Medical—Nursing Needs and Therapy Disciplines”) of the RCS-E v13.

In conclusion, the Italian RCS-E v13 appears to be a useful tool to assess clinical complexity in the Italian rehab scenario case-mix and its psychometric validation may have an important clinical rehabilitation impact allowing the assessment of the rehabilitation needs considering all three dimensions (disability, comorbidity and clinical complexity) as required by the Guidelines and the inhomogeneity could be reduced.

Introduction

According to the Italian National Plan for Rehabilitation (INPR) of 2011 [1], the Italian National Health Service (NHS) provides, in post-acute in-patients settings, for 3 levels of different Physical & Rehabilitation Medicine (PRM) interventions: intensive, intensive at high specialisation units and long-term or extensive care. Each Level should be based on the patient’s specific needs. A small number of people have very complex needs and require a higher level of highly specialised PRM care (e.g. post-acute spinal cord injury patients). A larger number of people also require specialist rehabilitation in similar settings, but in less specialised units (Intensive Level) for conditions such as some musculoskeletal trauma and neurological problems (e.g. Stroke).

In Italy, at present, especially at the Intensive rehabilitation level, the provision of rehabilitative services is not homogeneous and this is a well-known problem, as illustrated in an analysis of information flows disseminated by the Italian Ministry of Health with reference to the years 2010–2012 [2]. The report stresses the need to use appropriate criteria and parameters in the planning of rehabilitation actions. The same contents are also emphasised in Article 3 of the recent “Patto per la Salute 2014–2016” [3], reflecting the current problem of the growing economic impact of rehabilitation care and the cost-effectiveness analysis.

The INPR1 clearly specifies that Comorbidity, Disability and Clinical Complexity should be assessed to define the patient’s real needs and to plan the appropriate rehabilitation setting.

Whereas instruments such as the Barthel Index (BI) [4,5] and the Cumulative Illness Rating Scale (CIRS) [6] are typically used to rate disability and comorbidity, clinical complexity remains difficult to define.

Some European Countries, the US, Canada and Australia consider physical dependency and comorbidity also as indicators of clinical complexity of patients [7]. The available literature on rehabilitation suggests the use of various instruments to effectively evaluate Complexity in the rehabilitation domain [815]. Nevertheless, the scale that appears to be the most reliable and easy to use in order to evaluate the complexity dimension is the English Rehabilitation Complexity Scale (RCS).

The scale has been designed to provide a simple measure of rehabilitation complexity needs and interventions across the rehabilitation services and to determine the cost of the rehabilitation programmes delivered [16]

The preliminary version of the RCS16 assessed basic care, specialist nursing needs, therapy (in terms of time) and medical intervention, in order to take into account a description of the patient’s situation from a multi-professional (both a medical and a non-medical) point of view. A few years later the scale was revised [17] as RCS version 2, in which the therapy domain has been divided into two subscales reflecting the number of disciplines and the overall intensity of treatment [17]. The total score range was 0–15. However, clinicians working with the scale reported problems in identifying some aspects characterising patients with very complex needs, such as needs for special equipment. An Extended version of the RCS was developed to address these deficiencies [18]. The RCS-E version 12 [18] included 5 domains, Basic Care (0–4), Nursing Needs (0–3), Therapy (TD: 0–4, TI (0–4), Medical intervention (0–3) and Equipment (0–2) with a total score range from 0 to 20.

The latest version of the scale—is RCS-E version 13 –is currently used in the UK Rehabilitation Outcome Collaborative (UKROC) data set [19], a case-mix measure to strike a cost-effective balance between outcome and service cost, particularly for highly complex cases.

The RCS-E v13 encompasses the original 4 domains (C-N-M-T), which have been supplemented with a higher number of explanatory items of medical (0–4) and nursing (0–4) domains to assess higher levels of complex interventions. Furthermore, a new sub-domain “Risk” (0–4) was introduced to take account also of any need for supervision related to patients’ cognitive and behavioural aspects. Along with the Cure domain, a new domain called “Care or Risk” (0–4) was created. Where risk is measured, for the assessment both CARE and RISK shall be measured and the highest score shall be used.

Therefore, the English final version of the tool provides for 5 domains and has a score ranging from 0 to 22; it has an added value over the previous version in detecting patients with highly complex rehabilitation needs.

The RCS instrument proved practical and not time-consuming (administration time 5–10 min max). Furthermore, thanks to its psychometric properties [1718], it seems to be an excellent tool in supporting the decision to refer patients to rehabilitation. Recent studies reached the same positive conclusion validating the Danish RCS-E version 13 [20] and the Italian RCS-E version 12 [21].

The cross-cultural adaptation of the Italian RCS-E v13 also demonstrated that it is a sensitive and reliable tool that seems suitable for measuring clinical complexity in Italian intensive rehabilitation units, as reported in Rodà et al 2015 [22]. This preliminary study was designed in strict compliance with the relevant international guidelines, face validity and test-retest reliability were carried out to evaluate the comprehensibility and goodness of fit of the updated scale; it was conducted in three different rehabilitation units of the Emilia Romagna region (Northern Italy), involving 10 expert physicians and recruited 51 Intensive rehabilitation in-patients.

The aim of this study is to provide the first Italian RCS-E v13 validation by correlating the score of the scale with BI and CIRS, plus a list of appropriate admission criteria (see methodology) in three different categories of rehabilitation intensive in-patients [Neurological Patients (NP), Cardiological Patients (CP), Orthopaedic Patients (OP)] in order to support the use of the tool in Italy and to meet the INPR requirements.

Materials and methods

Study design, setting, sample and procedure

This is an observational multicentre prospective cohort study that did not change the everyday routine clinical practice but required collection of sensitive data for research purposes in compliance with the local Guidelines [23] and with the Declaration of Helsinki.

The study protocol was approved by the Bioethics Committee (CE) of the University Hospital of Parma (ref. CEPR-ProtN1175) and by the ethics committees (the ethics committee of the Reggio Emilia Province about "S. Sebastiano" Hospital of Correggio and IRCCS "Arcispedale Santa Maria Nuova" of Reggio Emilia; the internal ethics committee of the “San Giacomo” Hospital, Ponte dell’Olio, the internal ethics committee of the “Rizzoli” Orthopaedics Hospital, Bologna and finally the ethics committee of the NHS Local Agency of Piacenza for the Villanova d’Arda Hospital) of the centres taking part in the study. All the participants were informed of the implications of the study and signed an informed consent document before enrolling. All the relevant study documents were kept in the Inspector Site File and in the Trial Master File.

8 intensive rehabilitation facilities (Table 1) of various types of agencies, both National Health System (NHS) and private ones, as well as Research Hospitals, covering a large part of the Emilia Romagna Region (RER), participated in the research.

All the adult neurological, cardiologic and orthopaedic patients admitted to the intensive rehabilitation facilities from June 2014 to August 2015 were enrolled, for a total of 1712.

Before starting to collect the study data, at least one Rehabilitation Medicine Specialist (RMS) of the 8 participating rehabilitative facilities was trained in the administration of the experimental scales (RCS-E v13, CIRS, BI) and in the experimental procedures to ensure research efficiency. Statistical correlation analysis was then performed to assess training effectiveness.

Moreover, an electronic Case Report Form (eCRF)—GCP-compliant and compatible with the Italian NHS Information systems—was developed to collect the relevant experimental information.

Within the 72 hours after admission, i.e. the time required to evaluate the patients’ needs and plan the rehabilitation intervention, the trained RMSs administered the three scales (spending about 20 min for each patient: RCS-E v13: 5 min, CIRS 10 min, BI 5 min) and recorded the scores, together with the usual clinical information, in the medical records.

Three trained MD Detectors (MDDs), recruited specifically for this project and not involved in patient admission, were provided with laptops and authorised to move around the centres to enter the data from the medical records to the eCRF.

Variables/Measures

The variables used in the study were the Rehabilitation Complexity Scale Version 13 (RCS-E v13), the Barthel Index (BI), the Cumulative Illness Rating Scale (CIRS) and the Appropriate Admission Criteria (data not yet published).

The RCS-E v13 is a 22-point measure with 5 domains (see Rodà et al 2015): Basic Care or Risk (C or R: 0–4); Nursing (N: 0–4); Medical Needs (M: 0–4); Therapy Needs (Including the Number of Disciplines: TD and the Intensity: TI. TD+TI: 0–8); Equipment Needs (E:0–2). The higher the score, the more complex the needs are.

The BI is a 10-item ordinal scale that measures self-care, sphincter management, transfers and locomotion. Scores range from 0 to 100, in 5 steps, and the higher the score, the more independent the patient is.

The CIRS resulted in 14 categories, assessing the pathology and impairment of major organ systems and also psychological, metabolic, neurological and musculoskeletal aspects with a 0 (No problem) to 4 (Extremely severe problem) grading scale in each category. Several indices were derived from the CIRS: the total score (CTs) may theoretically vary from 0 to 56, although a high score is not compatible with life; the severity index (SI) is the mean of the scores of the first 13 categories, excluding the psychiatric one; and finally the comorbidity index (CI) as the number of categories with a score of 2 or greater, including the psychiatric one.

The list of Appropriate Admission Criteria in intensive rehabilitation facilities encompasses 14 items addressing functional ability/disability (e.g. mobility, self-managing, etc.), nursing care, medical and rehabilitation treatments needs, social and prognostic aspects (See S1 Appendix).

The list is the result of 6-month long stepwise work carried out through a consensus Delphi Panel Method [24]. The first step was collecting the admission criteria of the 8 centres involved in this study to prepare and provide the expert panellists with consecutive questionnaires, using a computerised process.

The first questionnaire consisted of 13 questions, some of which were broken down into sub-items, for a total of 29 items of a general nature. Patients were asked to use a five-point Likert scale to express their degree of agreement with the content of each item.

Then, their answers were collected and analysed (Cohen's Kappa correlation test) to identify shared and conflicting viewpoints. The convergence threshold for the answers is set to at least 70%, a value chosen on the basis of previous experiences described in literature [25,26].

12 items (41.4%) reached high convergence (at least 70%) at the first round. The items with a discordant outcome were re-written and represented in the subsequent questionnaire. The process gradually continued aiming at consensus synthesis (14 admission criteria) by the third round. (data not yet published).

Subsequently, the MDDs started the criteria validation as a Gold Standard through the PRUO (Protocollo di Revisione dell' Utilizzo dell' Ospedale) [27] procedure over 1 month (data not yet published).

The PRUO procedure [27], which was introduced for the first time in Italy in 1989 and subsequently adapted and adjusted, is a tool used to assess the appropriateness of the days of hospitalisation. It assumes that, in line with clinical complexity and consequent care needs, there is a hierarchy of healthcare levels based on the intensity of the intervention and on the services rendered. It involves a list of appropriateness criteria for each level of healthcare (which in the project is equivalent to the 14 admission criteria in Intensive rehabilitation facilities defined by the Panel). It also requires an Assessor to check the presence or absence of the criteria in patients’ medical records or other official hospital documents, considering the criterion to be satisfied (presence) when the wording of the criterion listed can be found in the patient’s documentation, in a univocally equivalent form.

The RMSs were blind to the appropriate admission criteria until the end of the research.

Statistical methods

Construct Validity: upon psychometric validation, the Spearman correlation (ρ) with 95% confidence intervals (CIs) was examined to compare the RCS-E v13 total score and subscales (C-R, N, M, T, E) to BI and CIRS, in order to evaluate case complexity in this context. The strength of the correlation was interpreted according to the guidelines suggested by Evans (1996): .00-.19_very weak, .20-.39_weak, .40-.59_moderate, .60-.79_strong, .80–1.0_very strong [28].

Concurrent Validity: due to the lack of Italian instruments to assess clinical complexity in rehabilitation patients as a gold standard, against which to test concurrent validity, the psychometric validity of the RCS-E v13 was evaluated comparing the 14 Admission Criteria in Intensive rehabilitation facilities (data not yet published) defined with the Delphi Panel Method. The Spearman correlation (ρ) was assessed.

Known-Group Validity: to control the confounding and to examine subgroups and interactions, the ANCOVA analysis has been performed considering as dependent variable the RCS-E v13 total Score, fixed factors, the Gender and Nosological Category and Covariate the Age.

The Kruskall-Wallis test was used to examine the variance between nosological sub-categories.

Dimensionality: the Kaiser-Mayer-Olkin measure and the Bartlett's test of sphericity were assessed to confirm the adequacy for factor analysis, using the AMOS-16 structural equation software programme within SPSS. Exploratory and confirmatory analyses, involving a principal component analysis with orthogonal (Varimax) rotation, were examined in order to evaluate the dimensionality of the scale. The inter-correlation between factors was studied.

The missing data were excluded from the analysis.

The software package “IBM SPSS version 22.0” and “MedCalc version 16.4.3” were used for analyses.

Results

The data were collected from 1712 Patients, 823 male (48%) and 889 female (52%), mean age 68.34 years old (95% CI 67.69–69.00 years old).

Of these, 1700 (520 NPR, 779 ORP and 401 CRP) were included in the statistical analysis. They were assigned to a unique primary nosological category, without any multiple classifications. See Table 2 for descriptive statistics and the Flow Chart in Fig 1.

thumbnail
Table 2. Demographic and clinical characteristic of patients statistically analysed (N = 1700).

https://doi.org/10.1371/journal.pone.0178453.t002

Considering the primary hypothesis, the 5% Type I error rate, the 20% Type II error rate and an expected correlation coefficient r (.07), the studied sample size ensures an 82.3% study power. (G*Power 3.1.9.2).

Table 2 shows the descriptive statistics for RCS-E v13, BI, CIRS of the 1700 patients included in the statistical analysis and assigned to a unique primary nosological category.

About the dimensionality results, all five domains of the RCS-E v13 administered to patients loaded “moderate” to “high” on the first unrotated principal component with loadings ranging from .43 to .72. Only the first two components had eigenvalue > 1, together accounting for 55.57% of the total variance in scores. Parallel Analysis indicated a two-factor solution which was rotated using a Varimax Procedure: the first factor appears to be “Basic Care or Risk—Equipment” which accounts for 31.38% of the variance. These items loaded high (.83-.86) on this factor and low (< .03) on factor 2. The second factor appears to be “Medical—Nursing Needs and Therapy Disciplines”, accounting for 24.19% of the variance. Medical loads .75 on this factor and -.27 on factor 1, Nursing loads .69 and .35, respectively, and, finally, Therapy Disciplines loads .69 on factor 2 and .09 on the other.

The results of the factor analysis suggested that the two-factor model had an excellent fit with the RCS-E v13 scores assessed, as showed with a goodness of fit index (GFI = .999), parsimony Good Fit Index (pGFI = .970) comparative fit index (CFI = .949) and the Bentler-Bonnet Normed Fit Index (NFI = .921). The good fit is confirmed because all indices are above .90 [27] Furthermore, RMSEA = .029 (C.I. 90% .005-.053 p-value .920) and Chi-Square = 9.82 (df = 4, p-value NA) also confirmed the model according to Barrett (2006) [29].

The covariance between “Basic Care or Risk—Equipment” and “Medical—Nursing Needs and Therapy Disciplines” factors was .12, supporting the different rehabilitation needs involved in each factor. The dimensionality results are reported in Table 3 and the path diagram in Fig 2.

thumbnail
Fig 2. Path diagram of the confirmatory factor analysis of the RCS-E v13.

https://doi.org/10.1371/journal.pone.0178453.g002

thumbnail
Table 3. Results of principal component factor analysis of the RCS-E v13 items.

https://doi.org/10.1371/journal.pone.0178453.t003

The results about Validity and relationship of the RCS-E v13 with the other measures are reported below. Known-Group Validity: the Kruskall-Wallis test showed the presence of a difference in the RCS-E v13 score among nosological categories (Chi-square 255.89, df 2, p < .001), as reported in Table 4. The ANCOVA showed, although with weak effect, a statistical relationship between the RCS-E v 13 total score, the nosological categories and the age of the sample (η2 .014). The mean value of the RCS-E v13 total score of CRP (mean age 69.7, SD ±11.46) was 9.18; the mean value of NRP (mean age 66.8, SD ±14.65) was 10.97; the mean value of ORP (mean age 68.6, SD ±14.65) was 9.67. These results suggest that being older does not affect the patients’ clinical complexity.

thumbnail
Table 4. Comparison of RCS-E v13 scores among the nosological categories of rehabilitation patients.

https://doi.org/10.1371/journal.pone.0178453.t004

Construct Validity: the RCS-E v13 reported strong negative correlations with the BI (rho = –.62, p < .001). Different correlations were assessed within the subscales, showing strong relation of C (rho = -.75 p < .001), moderate of E (rho = -.54 p < .001), weak of R (rho = -.21 p < .001) and N (rho = -.29 p < .001), and Very Weak of T (rho = -.05 p < .05) and M (rho = .09 p < .001). Furthermore, RCS-E v13 and BI correlations were different for different nosological categories. Correlation was found moderate for CRP (rho = -.58 p < .001) and ORP (rho = -.53 p < .001), and strong for neurological patients (rho = -.62 p < .001).

Weak correlation was found between the RCS-E v13 and the CIRS CTs, CI and SI scores, respectively (rho = .21, p < .001, rho = .21, p < .001, rho = .15, p < .001) and with little or no relation of some subscales like C, M and E, as reported in Table 5. The same results were obtained considering the nosological categories.

thumbnail
Table 5. Spearman’s correlation matrix of the RCS-E v13 scores to Admission Criteria, CIRS and Barthel Index.

https://doi.org/10.1371/journal.pone.0178453.t005

Concurrent Validity: the RCS-E v13 total Score demonstrated an overall correlation with most criteria, even if the rho resulted poor-modest (.05 to .27), thus confirming the concurrent validity of the tool. No statistically significant correlation was found between the RCS-E v13 score and criteria 2 (favourable rehabilitative prognosis), 3 (nursing access at least 3 times a day), 4 (daily need for medical intervention), 12 (hospitalisation due to the inability to move from the house to the gym) or 14 (treatments integration); these criteria were found always or almost always absent (2, 12, 14) or present (3 and 4), in the medical records. However, within the subscales, there were different correlations as reported in Table 5.

Discussion

The objective of this work is demonstrating the psychometric validity of the RCS-E v13 scale proposed to a sample of 1712 patients hospitalized in 8 different intensive rehabilitation centres of the Emilia Romagna Region. The statistical reliability of the pre-final version of the tool, cross-culturally adapted and investigated in accordance with the relevant international guidelines [30,31], was shown on a sample of 51 different patients previously investigated [22].

Currently, the absence, in Italy, of a shared tool for the evaluation of clinical complexity and the use of different evaluation tools among facilities have led to the proposal, to be defined through a scientifically-validated method, of the appropriate criteria for hospitalisation in intensive rehabilitation units that can be adopted as reference Gold standard (data not yet published). The Delphi Panel [2426,32] method was chosen. Through a Delphi Methodology, a national panel of experts in rehabilitation was called to examine the admission criteria of the 8 facilities participating in the project. The panel drew conclusions about the collected information and compared it to the literature in order to prepare a First Italian list of criteria (as standards) for appropriate admission to intensive rehabilitation facilities (data not yet published). The Panel produced a list of 14 general criteria (data not yet published) that can be adopted for all three nosological categories (NRP, CRP, ORP) studied (S1 Appendix). Criteria exploring all elements of clinical practice, such as the medical, nursing and time (3 months) areas, social aspects, rehabilitation/assistance planning and multidisciplinary path (n 5-6-7-8), would support the definition of tailor-made Rehabilitative Projects.

The list was then validated through the PRUO (Protocollo di Revisione dell'Utilizzo dell'Ospedale) [27] procedure. To minimise/prevent biases, ensure scientific validity of the collected data and their reliability to be used as a gold standard for admission to rehabilitation, randomization and blinding approaches were be used. The centres will be masked from the list of criteria (data not yet published) that will be known by the MDDs only.

Even though low, the correlation of these criteria with the RCS-E v13 seems suitable for investigation because, it can represent the relation between the patient’s complexity and the relevant rehabilitative objectives. The correlation of the subscales composing the RCS-E v13 and the differences between nosological categories seem to support this interpretation. Overall, these results have shown the concurrent validity of the tool. The Results of Known-Group Validity have shown that there are no statistical differences between nosological categories when the data are adjusted by age. The oldest patients are the CRP group who obtained the minor RCS-E v13 score. This shows that being older does not entail being clinically more complex. This unexpected result requires further investigations. The construct validity of the RCS-E v13 has been verified by correlating it with the other instruments, as described below. The comparison of the RSC.E v13 to BI suggests that the disability domain is a strong indicator of care needs, a moderate one for Equipment and a very week indicator of Medical and Therapeutic Needs, as resulted from the Original English Scale [17] and the Italian RCS-E v12 [21], retrospectively validated in the work by Galletti et al. [21]. Weak correlation was found with the newly added subscale Risk. The same general results have been confirmed for the neurological patients, while, for the cardiologic rehabilitative category, the disability degree, on average higher than the others, is a strong indicator of Care Needs only, a moderate one for equipment and a weak indicator for Risk. It is not an indicator of T and R in the orthopaedic patients. Weak correlation was found among RCS-E v13, CTs and the two indices SI and CI of the scale, which were even less correlated in the CRPs, as reported also in Galletti et al. [21], confirming the idea that probably CIRS is not a tool that directly measures the patient’s rehabilitative needs. These results appear to support the hypothesis that the comorbidity, expressed by the CIRS scale, summarised in the two indicators CI and SI, is not, therefore, a domain necessarily related to the clinical complexity of the rehabilitative patient. The factor analysis suggested that the Italian version of the scale has two different components (“Basic Care or Risk—Equipment” and “Medical—Nursing Needs and Therapy Disciplines”). This implies that each one of the scale 5 domains has a different impact in terms of rehabilitation requirements and the 5 domains together provide accurate profiling of rehabilitation needs.

Limits and strengths

In this research, some limits may be recognized:

A limit of the research is the inability to compare the scale to tools currently shared for direct evaluation of clinical complexity because, for the time being, they do not exist in the Italian rehabilitative scenario. However, this situation has generated the first national attempt to identify the shared intensive rehabilitation hospitalisation criteria.

It is also true, however, that the Gold standard defined by the National Panel was the result of the work of both clinical and administrative staff engaged in rehabilitation. The comparison to a totally clinical tool, such as the RCS-E v13 may have influenced the correlation indices that were not always found strong. However, the fact that the RCS-E v13 correlates with the criteria as well as with the BI and CIRS scales seems to support the validity of the tool as a clinical complexity indicator.

Another limit of this study is the case-mix investigated, which, in part, differed from the validation study of the original scale and from the pre-final 13 Italian version, in that it included also cardiological and orthopaedic patients. Nevertheless, the use of a homogeneous group of pathologies and the size of each one allowed the tool to validated for the measurement of rehabilitation complexity in patients having different rehabilitation needs, as previously demonstrated in Hoffmann et al. [33] and Galletti et al [21].

The study was conducted on only one intensity level of the rehabilitative hospitalisation, the intensive one. However, it is believed that the choice of a multicentre investigation, together with the involvement of patients belonging to different nosological categories for which disability and comorbidity are not necessarily equal, may be a strong point.

Another strong point is the perspective design of the study which could confirm that the ease and the speed in administering the RCS-E v13 make it a tool that can be effectively introduced in clinical settings.

Conclusions

Taking into consideration objectives, limits, range of analyses, similar results of other recent studies and other relevant evidence, we may conclude that version 13 of the Italian RCS-E has proved a tool that can be used for the evaluation of rehabilitative patients’ clinical complexity.

In order to evaluate the scale responsiveness and confirm its routine application in the national territory, further studies would be useful adopting administration not only at hospitalisation but also at discharge and in more levels of care intensity other than the intensive ones.

Efficient rehabilitation intervention requires rehabilitation settings that meet patient needs.

This study may provide a model for optimal use of rehab beds, for redefining reimbursement rates and assessing the required rehabilitation resources considering all three dimensions (disability, comorbidity and clinical complexity) as required by the Guidelines [1]. The 8 rehabilitative centres involved are representative of the Italian range of rehab facilities and patients; all this is expected to produce results fit to be used throughout the NHS to assess rehab needs.

Validated tools supporting referral to rehabilitation would make the admission process more appropriate and sharable. Furthermore, not only would they contribute in reducing national variability, but they would also improve clinical appropriateness.

Supporting information

S1 Appendix. List of Admission Criteria in intensive rehabilitation facilities.

https://doi.org/10.1371/journal.pone.0178453.s001

(DOCX)

Acknowledgments

We are grateful to all the teams (group name: GRECo) who worked to produce the data and to all patients that accepted to participate in the study. Special thanks are due to Professor Lynne Turner-Stokes, the Author of the scale, who provided version 13 of the RCS-E, as used today in the UK, and helped us with precious comments on the development of the Italian RCS-E v13 validation. We are also grateful to Dr. Luciana Bevilacqua and to Professors Francesco Auxilia and Silvana Castaldi of the Department of Biomedical Sciences for Health of the University of Milan, Italy, for having managed the Delphi Panel procedure and for having contributed to define, for the first time in Italy, the list of Criteria for Appropriate Admission to Intensive Rehabilitation Units, in full compliance with strict scientific methods (data not yet published).

Group name (Co-Authors): Emilia-Romagna Region Complexity Group (Italian acronym GRECo)

GRECo members and affiliations: Rodolfo Brianti, Rehabilitation Medicine Service, Rehabilitation Geriatrics Department of the NHS-University Hospital of Parma, Parma, Italy, on behalf of GRECo; Francesca Roda’, Department of Neuroscience, University of Parma, Parma, Italy, on behalf of GRECo; Maurizio Agosti, Rehabilitation Medicine Service, Rehabilitation Geriatrics Department of the NHS-University Hospital of Parma, Parma, Italy, on behalf of GRECo; Andrea Merlo, Motion Analysis Laboratory, Department of Rehabilitation, "S. Sebastiano" Hospital of Correggio, NHS Local Agency of Reggio Emilia, Italy, on behalf of GRECo; Maurizio Maini, “San Giacomo” Hospital, Ponte dell’Olio, Piacenza, Italy, on behalf of GRECo; Francesco Lombardi, Neurorehabilitation Service, "S. Sebastiano" Hospital of Correggio, NHS Local Agency of Reggio Emilia, Italy, on behalf of GRECo; Serena Maselli, Rehabilitation Medicine Service, Rehabilitation Geriatrics Department of the NHS-University Hospital of Parma, Parma, Italy, on behalf of GRECo; Elisa Corradini, Rehabilitation Nursing Home, Rehabilitation Department, Albinea Hospital, NHS Local Agency of Reggio Emilia, Italy, on behalf of GRECo; Davide Romiti, Department of Biomedical, Biotechnological and Translational Sciences, University of Parma, Parma, Italy, on behalf of GRECo; Sara Olvirri, Primary Care Department, NHS Local Agency of Parma, South/East District, Parma, Italy, on behalf of GRECo; Anais Rampello, Rehabilitation Medicine Service, Rehabilitation Geriatrics Department of the NHS-University Hospital of Parma, Parma, Italy, on behalf of GRECo; Giordano Ambiveri, Neurological Rehabilitation Medicine Unit, “San Giacomo” Hospital, Ponte dell’Olio, Piacenza, Italy, on behalf of GRECo; Emanuele Perotti, Orthopaedic Rehabilitation Medicine Unit, “San Giacomo” Hospital, Ponte dell’Olio, Piacenza, Italy, on behalf of GRECo; Marcello Giacani, Cardiological Rehabilitation Medicine Unit, “San Giacomo” Hospital, Ponte dell’Olio, Piacenza, Italy, on behalf of GRECo; Paola Marchetti, Neurorehabilitation Service, "S. Sebastiano" Hospital of Correggio, NHS Local Agency of Reggio Emilia, Italy, on behalf of GRECo; Claudio Tedeschi, Physical medicine and Rehabilitation Unit—Neuromotor Department, IRCCS "Arcispedale Santa Maria Nuova" of Reggio Emilia, Reggio Emilia, Italy, on behalf of GRECo; Stefania Testoni, Physical Medicine and Rehabilitation Unit, Neuromotor Department, IRCCS "Arcispedale Santa Maria Nuova" of Reggio Emilia, Reggio Emilia, Italy, on behalf of GRECo; Maria G. Benedetti, Physical Medicine and Rehabilitation Unit, “Rizzoli” Orthopaedics Hospital, Bologna, Italy, on behalf of GRECo; Silvia Bonfiglioli Stagni, Physical Medicine and Rehabilitation Unit, “Rizzoli” Orthopaedics Hospital, Bologna, Italy, on behalf of GRECo; Nino Basaglia, Department of Neuroscience and Rehabilitation, University Hospital of Ferrara, Italy, on behalf of GRECo; Fernando De Filippo, Rehabilitation Medicine Unit, Department of Neuroscience and Rehabilitation, University Hospital of Ferrara, Italy, on behalf of GRECo; Susanna Lavezzi, Severe Brain Injury Unit, Department of Neuroscience and Rehabilitation, University Hospital of Ferrara, Italy, on behalf of GRECo; Giovanni Delnevo, Extensive Orthopaedic Rehabilitation Unit, Department of Medicine, Borgo Val Di Taro Hospital, NHS Local Agency of Parma, Italy, on behalf of GRECo; Mara Contini, Extensive Orthopaedic Rehabilitation Unit, Department of Medicine, Borgo Val Di Taro Hospital, NHS Local Agency of Parma, Italy on behalf of GRECo; Domenico Nicolotti, Intensive Rehabilitation Medicine Spinal Unit, Emergency Medicine Department, Villanova d’Arda Hospital, NHS Local Agency of Piacenza, Italy, on behalf of GRECo; Monica Orsi, Intensive Rehabilitation Medicine Spinal Unit, Emergency Medicine Department, Villanova d’Arda Hospital, NHS Local Agency of Piacenza, Italy, on behalf of GRECo;

References

  1. 1. Ministero della Salute. Piano d’Indirizzo per la Riabilitazione Suppl. Ord. N60, G.U. n°50 del 2/3/2011.
  2. 2. Report Ministeriale 2010–2012. (2013) Ricoveri in Riabilitazione Ospedaliera.
  3. 3. Patto per la Salute 2014–2016 Rep n82/CSR del 10/07/2014, ai sensi dell’articolo 8, comma 6, della legge 5 giugno 2003, n. 131, tra il Governo, le Regioni e le Province Autonome di Trento e di Bolzano.
  4. 4. Granger CV, Dewis LS, Peters NC, Sherwood CC, Barrett JE. Stroke rehabilitation: analysis of repeated Barthel index measures. Arch Phys Med Rehabil 1979; 60(1): 14–17. pmid:420565
  5. 5. Galeoto G, Lauta A, Palumbo A, Castiglia SF, Mollica R, et al. The Barthel Index: Italian Translation, Adaptation and Validation. Int J Neurol Neurother 2015; 2: 0–28.
  6. 6. Salvi F, Miller MD, Grilli A, Giorgi R, Towers AL, Morichi V et al. A manual of guidelines to score the modified cumulative illness rating scale and its validation in acute hospitalized elderly patients. J Am Geriatr Soc. 2008; 56(10): 1926–1931. pmid:18811613
  7. 7. Jackson T, Dimitropoulosc V, Madden R, Gillett S. Australian diagnosis related groups: Drivers of complexity adjustment. Health Policy 2015; 119(11): 1433–1441 pmid:26521013
  8. 8. Troigros O, Béjot Y, Rodriguez PM, Shoaib F, Ellis H, Wade D. Measuring complexity in neurological rehabilitation: the Oxford Case Complexity Assessment Measure (OCCAM). Clin Rehabil 2014; 28(5): 499–507. pmid:24275452
  9. 9. Poulos C J, Magee C, Bashford G and Eagar K. Determining level of appropriateness in the patient journey from acute care to rehabilitation. BMC Healt Service Reserach 2011; 11: 1–9.
  10. 10. Poulos C J and Eagar K: Determining appropriateness for rehabilitation or other subacute care: is there a role for utilisation review?. Aust New Zealand Health Policy 2007; 4(3): 1–7.
  11. 11. Guilé R, Leux C, Paillé C, Lombrail P, Moret L. Validation of a tool assessing appropriateness of hospital days in rehabilitation centres. 2009; 21(3): 198–205.
  12. 12. Buntin MB. Access to postacute Rehabilitation. Arch Phys Med Rehabil, 2007; 88(11): 1488–1493. pmid:17964894
  13. 13. Buchanan JL, Andres P, Haley SM, Paddock SM, Young DC, Zaslavsky A. Final Report on Assessment Instruments for a Prospective Payment System. Santa Monica CA: RAND MR-1501-CMS; 2002
  14. 14. Huyse FJ, Lyons JS, Stiefel FC, Slaets JP, de Jonge P, Fink P, et al. "INTERMED": a method to assess health service needs. I. Development and reliability. Gen Hosp Psychiatry 1999; 21(1): 39–48. pmid:10068919
  15. 15. Boehlen FH, Joos A, Bergmann F, Stiefel F, Eichenlaub J, Ferrari S, et al. [Evaluation of the German Version of the "INTERMED-Self-Assessment"-Questionnaire (IM-SA) to Assess Case Complexity]. Article in German. Psychother Psychosom Med Psychol. 2016; 66(5): 180–186. pmid:27128827
  16. 16. Turner-Stokes L, Disler R, Williams H. The Rehabilitation Complexity Scale: a simple, practical tool to identify “complex specialised” services in neurological rehabilitation. Clin Med. 2007; 7(6): 593–599.
  17. 17. Turner-Stokes L, Williams H, Siegert RJ. The Rehabilitation Complexity Scale version 2: a clinimetric evaluation in patients with severe complex neurodisability. J Neurol Neurosurg Psychiatry 2010; 81(2): 146–153. pmid:19587391
  18. 18. Turner-Stokes L, Scott H, Williams H, & Siegert R. The Rehabilitation Complexity Scale—extended version: detection of patients with highly complex needs. Disabil Rehabil 2012; 34(9): 715–720. pmid:22115200
  19. 19. Turner-Stokes L, Poppleton R, Williams H, Schoewenaars H & Badwan D. Using the UKROC dataset to make the case for resources to improve cost-e ciency in neurological rehabilitation. Disability & Rehabilitation 2012; 34(22–23): 1900–1906.
  20. 20. Pedersen AR, Nielsen JF, Jensen J, Maribo T. Referral decision support in patients with subacute brain injury: evaluation of the Rehabilitation Complexity Scale—Extended. Disabil Rehabil. 2016; 6: 1–7.
  21. 21. Galletti L, Benedetti MG, Maselli S, Zanoli G, Pignotti E, Iovine R. Rehabilitation Complexity Scale: Italian translation and transcultural validation. Disabil Rehabil. 2015; 38(1): 87–96. pmid:25875050
  22. 22. Rodà F, Agosti M, Corradini E, Lombardi F, Maini M, Brianti R. Cross-cultural adaptation and preliminary test-retest reliability of the Italian version of the Complexity Rehabilitation Scale-Extended (13th version). Eur J Phys Rehabil Med 2015; 51(4): 439–446. pmid:24621987
  23. 23. Autorizzazione generale al trattamento di dati personali effettuato per scopi di ricerca scientifica—G.U. N. 72 del 26-3-2012.
  24. 24. Dalkey NC & Helmer O. An experimental application of the Delphi method to the user of experts. Management science (1963; 9(3): 458–467.
  25. 25. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 2000; 32 (4): 1008–1015 pmid:11095242
  26. 26. Linstone HA, Turoff M. Introduction in: The Delhi method: techniques and applications. Turoff Murray and Linstone Harold A. (eds.)–MA: Addison-Wesley publishing company; 1975.
  27. 27. Castaldi S, Bevilacqua L, Arcari G, Cantù AP, Visconti U, Auxilia F. How appropriate is the use of rehabilitation facilities? Assessment by an evaluation tool based on the AEP protocol. J Prev Med Hyg. 2010; 51(3): 116–120. pmid:21361116
  28. 28. Evans JD. Straightforward statistics for the behavioural sciences. Pacific Grove: Brooks/Cole Pub. Co., 1996.
  29. 29. Barrett P. Structural equation Modelling: Adjudging model fit. Personality and Individual Differences 2007; 42: 815–824.
  30. 30. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol 1993; 46: 1417–1432. pmid:8263569
  31. 31. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report meas- ures. Spine (Phila Pa 1976) 2000; 25: 3186–3191.
  32. 32. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ, 1995; 311(7001): 376–380. pmid:7640549
  33. 33. Hoffman K, West A, Nott P, Cole E, Playford D, Liu C et al. Measuring acute rehabilitation needs in trauma: preliminary evaluation of the Rehabilitation Complexity Scale. Injury. 2013; 44(1): 104–109. pmid:22130452