Aims and hypothesis

The aim of this study was to calculate the test–retest reliability of the SPPB, OLST, and TUG test and to calculate their absolute reliabilities with the standard error of measurement (SEM) and MDC scores at the 90% confidence interval (MDC₉₀) threshold.

Design

This was a prospective, nonexperimental, and descriptive research study.

Setting and participants

The participants were recruited from two HD units in Valencia and one unit in Barcelona (Spain) between 2013 and 2015. All the participants were explained the protocol and the procedures to be used, and signed their written informed consent prior to participation. This study was approved by the Ethics Committee at the Hospital Universitario Doctor Peset and is registered at ClinicalTrials.gov (reference number NCT02830490). The attending nephrologist reviewed and authorized their patients’ potential inclusion before the subjects were approached to solicit their interest. Patients were included in the study if they had been receiving maintenance HD for at least 3 months and did not have any acute or chronic medical conditions that would preclude the collection of the test data; they were excluded if they had recently had a myocardial infarction (within 6 weeks), unstable angina, malignant arrhythmias, or any disorder that was exacerbated by activity. The following demographic and clinical data were collected from the patients’ medical histories: age, sex, body mass index, time on HD, creatinine, albumin, and hemoglobin levels, cause of kidney disease, and the Charlson Comorbidity Index score.

Procedure

Participants performed the SPPB, OLST, and TUG tests twice, with an interval of one to two weeks between the testing sessions (test–retest evaluation research format), always immediately before the first HD session of the week. Every effort was made to maintain consistency between the testing sessions, including control of factors such as the day of the week, time of day, testing area, and the person conducting the assessment, although not all the subjects could be assessed in both sessions. At the two HD units in Valencia, two different physical therapists (researchers 1 and 2) with 11 and 8 years’ experience in physical function evaluation, respectively, performed and assessed the tests; a renal nurse with 5 years’ experience in evaluating physical function assessed the participants at the third HD unit in Barcelona.

Short physical performance battery.

The SPPB objectively measures lower extremity function, including performance-based balance, endurance, and strength. Each component is scored from 0 to 4 and summed to yield scores between 0 (poor) and 12 (best) performance [5] (Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Short physical performance battery scoring.

https://doi.org/10.1371/journal.pone.0201035.t001

To test standing balance, the participants were asked to maintain their feet in the side-by-side, semi-tandem (heel of one foot beside the big toe of the other foot), and tandem (heel of one foot directly in front of the other foot) positions for 10 seconds each. In order to test endurance, we asked the subjects to walk for four meters at their normal pace. Participants were allowed to use their usual walking aid, although they were encouraged not to use it, and were scored according to the quartiles for the length of time required. Lower limb strength was tested by asking the subjects to fold their arms across their chests while standing up and sitting down five times (STS-5) as quickly as they could. The chair used for the test had no armrests and was backed up against a wall to minimize the risk falling. A stopwatch recorded the time taken until the peak of the fifth rise [23,24].

Statistics

Normally-distributed descriptive data are reported as the mean plus the standard deviation (SD), or otherwise, as the median plus the range. The Kolmogorov–Smirnov test was used to assess the normality of the data. We also performed paired comparisons with the paired t-test or the Wilcoxon signed rank test to assess any systematic bias between the trials. The ICC (model alpha) and a two-way random-effects model were used to assess the test–retest reliability of the data for all the repeated tests; we considered an ICC above 0.75 to demonstrate good reliability, although for clinical measurements it has been suggested that the ICC should exceed 0.90 [32]. The SEM was used to determine the absolute reliability of the tests and represents the extent to which the outcome can vary in the measurement process. It was calculated with the following formula:

Where r is the ICC for the participant groups.

The MDC is defined as the amount of change in a measurement required to conclude that the difference is not attributable to error; it is the smallest change that falls outside the expected range of error thus, any change exceeding the MDC₉₀ is considered genuine and indicates confidence in the test’s predictive abilities [4,27,33,34]. The MDC₉₀ was computed from the SEM with the following formula:

A Bland-Altman plot of each participant’s mean score (SPPB, OLST, TUG) plotted against their difference score (trial 1-trial 2) was constructed to display the spread of difference scores about the mean difference score. The Bland-Altman plots also display the 95% limits of agreement (95% LOA) which represents the expected range of difference scores across trials of the tests. The 95% LOA was calculated as the difference in mean scores of the tests ± SD x 1.96, with the SD as the standard deviation of the difference scores.

Correlation between the three tests and hemoglobin, albumin and creatinine was explored thouth the Spearman correlation coefficient.

We set the level of significance required to a probability of P ≤ 0.05 for all our statistical analyses. The data were managed and analyzed using the Statistical Package for Social Sciences (SPSS) version 20.0 for Windows.

Test–retest reliability

The SPPB examines three areas of lower-extremity function (static balance, gait speed, and getting in and out of a chair) that are representative of essential tasks for independent living among CKD patients on HD. The SPPB is useful for predicting outcomes such as falls, institutionalization, and death in elderly population [5], and although it has previously been applied to HD patients [6,35], to our knowledge, ours is the first study describing the relative reliability of the SPPB in patients undergoing HD. Our results showed that this test has excellent test–retest reliability (ICC = 0.94; 95% CI = 0.91–0.97), and are consistent with values reported for a community-dwelling older population (ICC = 0.82; n = 487; mean age 74.1±5.7 years) [36] and for older women (ICC = 0.88–0.92; n = 1002; mean age 78.3±0.3 years) [37].

Similar to our study, Studenski et al. [36] performed the test–retest after one week, although they used a different testing site between trials: first during an outpatient clinic visit and then as part of a comprehensive home visit. In our case, we acquired all the measurements for both trails at the same location and within one or two weeks. In our study the ICC for the SPPB was high, suggesting that it is a good physical performance test for identifying loss of mobility in CKD patients undergoing HD. Future longitudinal studies should clarify whether the SPPB can predict difficulties in the activities of daily living in HD patients, as it can in elderly and older hospitalized patient populations [37,38].

No previous studies have reported the relative reliability for the OLST in patients undergoing HD, although the OLST ICC values reported in other populations are generally lower than our results (ICC≥0.90). In elderly populations the ICC ranges from 0.60 [8] to 0.86 [11], following hip fracture it was 0.75 and 0.83 in the affected and non-affected leg, respectively [12], and it was 0.87 for patients with a lower-limb amputation [9]. In contrast, an ICC of 0.994 was reported for a subgroup of 50 healthy military health-care beneficiaries aged 18 and older.

There are a wide variety of published protocols for performing the OLST, but surprisingly little consensus regarding how it should be conducted. For example, some studies use a maximum time of 10 seconds [39,40], and others 30 seconds [8,12,41], 45 seconds [13,39], or 60 seconds [9,11,42]. We chose to use 45 seconds as maximum time because Briggs et al. [10] posit that a limit of 45 seconds results in normal data distribution [10,13]. Another variable is the number of attempts the patient is allowed to achieve the maximum time: while some studies do not report this factor [8,12], in other trials it ranges between three [39,41,42] and five [9,11]. Additionally, some authors use the average of the trials for their statistical analyses [11,39] while others use the single longest time achieved [9,10,42]. Following the procedure published by Hurvitz et al. [13], itself based on Briggs et al. [10], we performed three trials and used the longest time achieved for our data analysis. This strategy appears to provide a good indication of balance capabilities because the best trial results were almost always obtained among the first three test trial results [10,13].

The details of how the OLST studies are executed also often differ: as in other studies we allowed our participants to keep their eyes open [9,39,41,42], wear shoes, choose the leg they preferred for the test, and to move their arms to help maintain their balance [13]. Moreover, our sample size was larger than that of previous studies (n = 62) and the ages included ranged from 21 to 90 years (mean 61.4±16.4 years), making ours a relatively young sample compared to other studies (see Table 5). Future studies should aim to assess if the OLST is useful for predicting falls in CKD patients.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Characteristics of selected studies which use the one-legged standing test.

https://doi.org/10.1371/journal.pone.0201035.t005

The TUG test is a validated and commonly used method for assessing functional mobility; its relative reliability values have been reported in different populations including elderly (ICC = 0.98–0.99) [15,25], chronic heart failure (ICC = 0.93) [16], Parkinson disease (ICC = 0.80) [26], and Alzheimer disease (ICC = 0.985–0.988) [27] cohorts. Our results showed that the relative reliability of this test for patients undergoing HD is excellent (ICC = 0.96), therefore suggesting that this is an appropriate test for assessing this aspect of physical function in CKD patient groups. Additionally, this was the only test that correlated with the inverse creatinine values of the sample.

Taken together, our findings demonstrate that test–retest reliability for the SPPB, OLST, and TUG clinical tests was excellent. Factors that might explain these good results, and that should therefore be considered in the application these tests in clinical environments, include performing these tests (i) before a HD session, (ii) on the same day of the week, and (iii) after adequate research training and standardization of the assessors’ instructions. However, it is surprising that the relative reliability (ICC) in a sample with such high comorbidity (CKD patients on maintenance HD) was higher than in other cohorts with, presumably, lower health status variability (e.g. elderly populations with no chronic disease). This could mean that young people receiving renal replacement treatment are usually in a better physical condition than elderly populations receiving HD, leading to the increased consistency seen in the former in this present study.

Another reason could be the uniformity of our protocol which we designed to ensure standardization, both of the procedures and between the researchers performing the tests. Our testing instructions were the result of a consensus between the different research teams at each center undertaking the study. Surprisingly, our review of previously published studies regarding functional testing, revealed inconsistencies between the testing protocols used across a variety of tests, including the OLST. These factors might lead to inappropriate results being reported and may hinder meaningful comparison between the outcomes of different studies. Thus, we believe it is very important that both researchers and clinicians assess physical functioning in future studies using the same tools and by implementing standardized instructions.

Minimal detectable change

Despite the excellent test–retest reliability results for our patient cohort, the performance of individual participants between sessions still substantially varied, producing high MDC values (Table 4). The MDC₉₀ is the threshold of change that a measurement must reach in order to exceed the anticipated measurement error and variability, and is a conservative estimate of clinically meaningful score changes. In this case, the magnitude of clinically meaningful change in these physical performance tests can help clinicians and researchers to identify important functional changes in CKD patients undergoing HD [4]. The MDC for the SPPB, OLST, and TUG test have been previously studied in other populations including the elderly [5,8,11,43], people recovering from a hip fracture [12] or lower-limb amputation [9], and in groups with Alzheimer disease [27]. Nevertheless, to our knowledge, this is the first study to calculate the MDC of these tests in patients with CKD undergoing HD.

Our results produced an MDC₉₀ of 1.7 points for the SPPB, whereas in an elderly population, a change of one point was representative of a meaningful difference in the risk of future mortality and the incidence of disability [5]. Another large study of older adults (n = 482; mean age 74.1±5.7 years) reported a SEM of 1.42 points [44], compared to the SEM of 0.72 points we obtained in this study. In this case, the time frame of the test–retest assessment was longer than in our study: the subjects were evaluated at the participant’s house every three months for the first year and every 6 months for the second year. In our study we strictly replicated all the measurement conditions, but even so, the physiological and clinical status of patients undergoing HD can widely vary, potentially leading to heterogeneity in the results.

Our OLST results gave an MDC₉₀ of 11.3 s, whereas in a community-dwelling population, the MDC₉₅ was 24.1 s [11]. This, perhaps surprising difference can be explained by the high SD in the latter study sample (20.4 s) [45]. In patients with a lower-limb amputation the MDC₉₅ was 2.74 s [9], and this difference can also be related to the evaluation procedure: while we performed three trials with a maximum time of 45 seconds, other studies performed five trials with a maximum time of 60 seconds [9,11]. We chose three rather than five trials to try to achieve the longest time possible (in the knowledge that the best score is usually obtained in the first three trials), while also aiming to reduce variability and to avoid muscle fatigue [10].

The MDC₉₀ for the TUG test in this present study was 2.9 s. In comparison, the MDC₉₅ in a cohort with Parkinson disease was 3.5 s [26] (similar to our results if we calculate the MDC₉₀) and in another sample with Alzheimer disease, the MDC₉₀ was 4.09 s [27]. The high MDC found in the Alzheimer disease study can be explained by its high SD (19.95 ± 9.81 s in mild-moderate disease and 28.01 ± 17.49 s in moderate-severe to severe disease); patients with a higher level of dementia produce more variable results and need more time to perform the test compared to less demented subjects, thus generating higher MDC scores. Another important difference is the number of trials performed: while we carried out three trials, Ries at al. [27] performed two trials in patients with Alzheimer disease and Huang et al. [26] only measured the TUG once, so as to avoid fatigue (although they concluded that more trials would increase the stability of the measurement and would reduce its MDC). Hence, performing more than one trial increases the stability of the test, and as a result, decreases the MDC.

In summary, the MDC₉₀ results that we obtained in this study (1.7 points for the SPPB, 11.3 s for the OLST, and 2.9 s for the TUG test) represent the threshold-change values required to be 90% certain that any changes noted in the test results for any given individual patient are not due to internal variability. In the clinical field, researchers and clinicians should use these MDC values to determine whether differences in the test results obtained between follow-up trails in their CKD patients on maintenance HD represent true changes which may be associated with poor prognosis.

Study limitations

The main limitation of this study was the variability of our cohort in terms of its broad sample age range which may have introduced error related to the probable increased presence of comorbidities in older patients. It is also worth noting that the patient participation rate was low. Additionally, we did not register interdialytic weight gain between the first and the second evaluation day, though we tried to keep all other factors stable (HD session of the week, time, assessor). Moreover, only 30 minutes were available to perform these assessment tests before the HD session started which may have led us to rush in some cases. However, despite this time constraint, we tried to limit extrinsic variation by following a strict methodology. Another potential limitation to inter-study comparisons is the lack of academic consensus on the exact OLST testing procedure.