https://orcid.org/0000-0002-4748-5514
Figures
Abstract
Background
The current clinical gold standard for assessing isometric quadriceps muscle strength is an isokinetic dynamometer (IKD). However, in clinics without an IKD, clinicians default to using handheld dynamometers (HHD), which are less reliable and accurate than the IKD, particularly for large muscle groups. A novel device (ND) was developed that locks the weight stack of weight machines, and measures forces applied to the machine, turning this equipment into an isometric dynamometer. The objectives of this study were to characterize the test-retest reliability of the ND, determine the within-day and between-days inter-rater reliability and concurrent validity compared with that of the HHD, in healthy volunteers (HV) and individuals with knee osteoarthritis (OA) for measuring knee extensors isometric muscle force.
Materials and methods
29 healthy (age = 28.4 ± 7.4 years) and 15 knee OA (age = 37.6 ± 13.4 years) participants completed three maximum force isometric strength testing trials on dominant side knee extensor muscles on three devices (ND, HHD, and IKD) in two separate sessions by two raters. The maximum force (Fmax) produced, and the force-time series were recorded. Reliability and validity were assessed using Intraclass Correlation Coefficient (ICC), Bland-Altman Plots, Pearson’s r, and cross-correlations.
Results
The ND demonstrated excellent test-retest reliability (ICC2,3 = 0.97). The within-day (ICC2,3 = 0.88) and between-day inter-rater reliability (ICC2,3 = 0.87) was good for HHD. The ND showed excellent within-day (ICC2,3 = 0.93) and good between-day (ICC2,3 = 0.89) inter-rater reliability. The Bland-Altman analysis revealed HHD systematic bias and underestimation of force particularly with quadriceps force values exceeding 450 N. Mean differences were found in maximum force between HHD vs. IKD (MDabs = 58 N, p < .001) but not the HHD vs. ND (MDabs = 24 N, p = .267) or ND vs. IKD (MDabs = 34 N, p = .051). The concurrent validity of Fmax (r = 0.81) and force-time curve correlation (0.96 ± 0.05) were the highest between the ND and IKD.
Citation: Sahu PK, Goodstadt N, Ramakrishnan A, Silfies SP (2024) Test-retest reliability and concurrent validity of knee extensor strength measured by a novel device incorporated into a weight stack machine vs. handheld and isokinetic dynamometry. PLoS ONE 19(5): e0301872. https://doi.org/10.1371/journal.pone.0301872
Editor: Žiga Kozinc, Faculty of Health Sciences, University of Primorska, SLOVENIA
Received: December 29, 2023; Accepted: March 22, 2024; Published: May 22, 2024
Copyright: © 2024 Sahu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The dataset used and/or analyzed during the current study and description and development of the device are available on the Center for Open Science https://doi.org/10.17605/OSF.IO/E4GWF.
Funding: This publication was made possible in part by the Drexel University ExCITE Seed Funding and a grant through the Coulter-Drexel Translational Research Partnership Program (002537). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: We have read the journal’s policy and the authors (ND, AR, SPS) of this manuscript have the following competing interests: A United States patent through Drexel University for the novel device used in this study (Portable Load Testing Device, Patent. No. US 11,099,089). This does not alter our adherence to PLOS ONE policies on sharing data and materials.
List of abbreviations: CI, Confidence Interval; Fmax, Mean maximum force; GUI, Graphical user interface; HHD, Hand Held Dynamometer; HV, Healthy volunteers; ICC, Interclass Correlation Coefficient; IKD, Isokinetic Dynamometer; LoA, Limits of Agreement; MDD, Minimal detectable difference; MDabs, Mean Absolute Difference; N, Newtons; ND, Novel Device; NPRS, Numerical Pain Rating Scale; OA, Osteoarthritis; SDpooled, average standard deviation; SEM, Standard Error of Mean; THR, Total Hip Arthroplasty; TKR, Total Knee Arthroplasty;; US, United States
Introduction
Knee joint injuries account for 39.8% of all sport and 60% of high-school sport-related surgeries in the United States (US) [1, 2]. Quadriceps strength is critical in preventing knee injuries and degenerative joint disease by attenuating ground reaction force [3, 4]. Quadriceps strength impairment is common in osteoarthritis (OA) and post-knee surgery, with deficits that can persist even for years, impacting the ability to perform lower extremity function and sports activities [4–6]. Additionally, longitudinal data indicates quadriceps strength is a robust predictor of functional decline and mortality due to coronary artery disease and chronic obstructive pulmonary disease [7, 8]. Studies showed a positive association of quadriceps strength with dynamic balance, gait speed, stride length, fall risk, general health, and function [9–13]. Evaluation of peak muscle force (strength) is an essential clinical component of assessing muscle function [14]. Moreover, quadriceps strength assessment helps quantify strength deficits, intervention program effectiveness, progression through rehabilitation evaluation for return to sport, and development of normative data [12, 13, 15–17].
Objective strength measurement tools used in clinical practice vary from hand-held dynamometers (HHD) to computerized isokinetic dynamometry (IKD). Utilizing any device on a large scale for clinical practice depends on several factors, such as reliability, validity, affordability, safety, and portability [18]. Reliability evaluates the consistency of repeated measurements, and test-retest reliability specifically addresses consistency across measurement sessions using either the same (intrarater) or different (interrater) clinical testers [19]. Validity is defined as how well the device measures what it intends to measure. Concurrent validity assesses how well a new device measurement matches an existing criterion.
In clinical practice, muscle strength is often assessed with a HHD [20]. When evaluating knee extensor strength, the examiner typically stabilizes the HHD against the distal anterior tibia, (manually or assisted by a belt) to resist the participant’s active knee extension and record the highest amount of force produced during the test [21]. HHDs are portable, inexpensiveness (∼$1000), and user-friendly, which supports their use in physical therapy, sports, and fitness industries. When larger muscle groups are assessed, HHD application requires changes in practice to account for greater stabilization requirements (for muscles producing more than 15–30 kg force), and alteration in examiner or client position to reduce inconsistency and inaccuracy [21, 22]. Variability in measurement practice is especially problematic for large muscle groups such as knee and hip extensors, as forces produced can reach or exceed 500 newtons [23]. These larger forces rely on the tester’s ability to manually resist a patient-generated force; thus, during the use of a HHD, the tester’s ability can affect both the reliability and validity of an isometric force reading [21, 24, 25]. Studies have reported differences in maximum force measures of knee muscle extensors and flexors between male and female testers, with the study by Kelln et al. reporting lower reliability with female testers compared to males [26, 27]. To eliminate the issues related to HHD use, the current gold standard for assessing large muscle group isometric strength is the IKD. IKDs are large pieces of equipment that control body segment stabilization, positional consistency, and resistance without the need for the assessor to provide them. Research documenting the reliability and validity of these devices makes them the gold standard for strength testing [28, 29]. Additionally, IKD can measure various variables, including torque, power, and velocity, and the capacity to maintain patient data over an extended period [30]. While a number of companies produce these devices (e.g., Biodex, Computer Sports Medicine Inc., BTE Technologies), they are expensive (>$40,000) and the associated space requirements restrict the availability of IKDs and often limit their use to research and elite sports training facilities [24, 31]. The IKD can be cumbersome to set up for testing, and training is needed for the efficient operation of the device, further challenging its application.
Reported intrarater reliability is moderate to good using conventional HHD protocols for assessing knee extensor muscle strength (examiner provided resistance against a client-generated force). However, interrater and test-retest reliability, an indicator of a device’s practical use in a clinical setting, is unclear, and significant variations are detected in strength measurement between testers and sessions [27, 32, 33]. Prior work has modified HHD protocols to reduce the influence of the tester’s ability to generate an adequate counterforce. Approaches include the use of belts looped around the back leg of the chair to anchor the HHD [34]. Other studies focused on controlling the influence of the tester within the protocol to reduce the error created by poor stabilization of the person or position (providing support by belt and chair frame) [35–38] and leveraging resistance (dynamometer mounted either on a PVC pipe or the back of the chair with a belt) [39, 40]. However, these controlled studies rarely address interrater or test-retest reliability. Without between-day test-retest reliability, it is impossible to objectively determine if a client is truly improving their strength in response to an intervention.
Although the modified HHD and force transducer systems have several advantages related to the costs, the major drawbacks in those studies were poor fixation of the HHD pad on the tibial segment [35–44] and lack of proximal stabilization [36–40, 42, 45–47]. Further, many HHDs do not provide feedback during testing [35, 37–41, 43–46], are time-consuming for setting up participants [35, 36, 42], and are not completely tester hands-free measurements [35, 38–41, 43]. A few modified HHD systems are not portable [35, 36, 41, 42, 46, 47], and measuring the strength of other large muscle groups is not provided by the equipment [35, 41, 45–50]. During strength testing, the contact surface of the pad on the tibia may produce pain or discomfort, as reported in a paper [34]. Therefore, it may produce lower force, especially in ongoing painful individuals.
HHD validity studies compared to IKD report correlation values from .43 to .99 across arm and leg muscles, with inconsistent evidence for validation for larger muscle groups [51, 52]. Experiments conducted using modified HHD protocols (external stabilization of the device or person) also provide conflicting results, with some studies indicating that HHD provides a lower force value than IKD, while another study reports overestimation [37, 43]. Despite varying protocol modifications, no standardized protocol has been established that allows clinicians to implement an optimal test efficiently for larger muscle groups. The discussion of limitations of the use of HHD noted in prior studies, particularly when measuring large muscle group strength, suggests the need for adaptations to testing that allow for more consistent setup, stabilization, and measurement or the necessity for the expensive IKD.
To this end, a novel isometric strength testing device (ND; Patent US 11,099,089) was constructed to address the limitations of the HHD and the current gold standard (e.g., cost, portability, size) while providing reliable and valid strength measures of larger muscle groups. The ND locks down any weight stack, converting it into an isometric system and allowing measurement of the forces applied by the individual through the weight stack cable. Weight stack machines provide improved customization, standardization, and stabilization for individuals during strength testing and eliminate tester manual resistance, and stabilization errors (see the Materials and methods section for details).
Therefore, the first objective of this study was to determine the within-day test-retest reliability of the ND to assess knee extensors’ isometric muscle strength in healthy volunteers (HV) and individuals with knee osteoarthritis (OA). The second objective was to find the within-day and between-day interrater reliability of the ND, and to compare it with that of the HHD (with rater manual resistance and participant stabilization). The third objective was to determine the concurrent validity of the ND against the gold standard (IKD) and differentiate that from the concurrent validity of the HHD.
Materials and methods
Participants
This was a prospective observational study conducted at Drexel University. The detailed protocol was approved by the University Institutional Review Board for human subjects (IRB 1610004896), and the participants provided their written consent after discussing potential risks. The sample size was calculated based on the formula reported by Walter et al. for reliability studies (Online calculator-https://wnarifin.github.io/ssc/ssicc.html) [53, 54]. To demonstrate the test-retest reliability of ND with expected reliability (ICC = 0.85) and a confidence interval (CI = ± .1), 80% statistical power, and two measurements, we calculated the minimum sample size required as 31. We estimated the test-retest reliability of ND to be at least 0.85 as published studies using an HHD demonstrated the intrarater, test-retest ICC of knee extensors is 0.78. We believed the ND would surpass this ICC value [21]. We adjusted the sample size needed for this study to 33 participants to account for a 10% dropout rate.
Flyers and posters in the local community and at the university were used to recruit healthy participants (no knee injury or pain) and those diagnosed with knee OA. Participants were recruited between January 3 and October 6, 2017. An online survey, email, and telephone interviews were conducted by the investigators to confirm the participant’s eligibility. The inclusion criteria for participants were adults between 18 to 60 years old. The healthy participants had to report no lower extremity injury and no diagnosis of knee or hip OA. Participants with knee OA had a diagnosis of mild to moderate knee osteoarthritis in their dominant leg, were able to walk and climb stairs without an assistive device and had no activity restriction by a physician (cleared for exercise). Potential participants in both groups were excluded if any current or previous spine or lower extremity injury, that significantly affected their daily activities, or recent (last three months) knee or hip surgery on the dominant lower extremity. Other exclusion criteria were neurological deficits, altered sensation, diagnosed depression or anxiety, fatigue disorders, or chronic widespread pain that could affect force production and fatigue of the muscle.
Novel device
The ND consists of structural and electronic elements (Fig 1). Structural elements are composed of sliding telescopic struts that form the vertical post on which we fixed a 6” right angle foot on the bottom that slides under the weight stack machine’s frame. A 1” right angle bracket is secured on the top telescoping strut in which we mount the S-beam load cell (AmCells STL-500lb, tacunasystems.com). An eyebolt was fitted on the top of the load cell, which served as the guide for the pin that locked the weight stack. The electronic components consist of a load cell amplifier (HX711 with 5V excitation) connected to a microcontroller (ATmega328P Nano Board, ARDUINO). Custom firmware was programmed into the microcontroller, allowing it to read the sensor output, convert it to newtons, and transmit it via Bluetooth (HC06) RS232 serial port protocol to a laptop running our custom software. To power these electronics, our initial design used an external 5V rechargeable battery pack, which was later swapped with a 7.4V internal Li-Poly battery and charger pack. A custom graphical user interface (GUI) program running on the laptop/tablet was written in LabVIEW (version 8.6, National Instruments, Austin, TX) to connect, display, and save real-time data. A PC application was developed that added functionality, including subject profile, report generation, and several graphing improvements. For additional information regarding the development and design of the device, see https://doi.org/10.17605/OSF.IO/E4GWF.
The left figure shows the configuration of different parts of the novel device, as described above. The device can be anchored under the bottom of the steel frame of any commercially available weight machine (right bottom figure). The top of the strut has an eyebolt through which the weight selector pin passes and is set at the highest weight stack (right top figure). This locks down the weight stack, converting it into an isometric system and allowing measurement of the forces applied by the individual to lift the weight stack. The load cell registers the application of a tension force to the steel pin. The force is sensed by the digital-to-analog microcontroller attached opposite the load cell. Real-time force data, including the force-time graph from the microcontroller, can be wirelessly displayed and recorded by associated custom software.
Procedures
Fig 2 provides an overview of the measurement protocol, which was conducted in two separate sessions 3–14 days apart. The total time needed to complete each session was approximately 1.5 hours. Session one included: assessment of demographics and anthropometric data [age, sex, height (cm), weight (kg), thigh and lower leg segment lengths (cm), and leg dominance (the leg used to kick a soccer ball with maximal effort)]. Participants completed the Saltin-Grimby Physical Activity Level Scale (SGPALS) questionnaire [55]. This is a 4-level scale that measures leisure time physical activity, with the lowest score ‘of 1’ representing “physically inactive” and the highest score ‘of 4’ representing “regular hard physical training for competitive sports.” This tool previously showed a positive association with cardiovascular risk factors, mental health, and predictors of cerebrovascular diseases [56–58]. This self-reported scale has demonstrated good reliability and validity in a large-scale adult population [59]. The participants with knee OA were also asked to note their side of pain and the intensity of pain recorded using an 11-point numeric pain rating scale (NPRS) [‘0’ corresponding to “no pain” to ‘10’ represents “maximum intensity of pain”]. This scale has demonstrated acceptable ranges of reliability [60]. During each session, raters collected dominant leg knee extensor muscle strength using the HHD and ND with device testing and rater order randomized a priori.
This is a schematic of the two-session testing protocol, with an indication of the datasets used for each analysis.
In session one, the second rater completed a second measurement series using the ND to obtain the ND’s test-retest reliability. The strength measurement using the IKD was always performed in the second session and by the second rater after the measurements using the ND and HHD were completed. Two licensed physical therapists and one exercise scientist (three males) were trained as testers on all three devices. All testers were familiar with and used HHD and IKD testing devices.
Pre strength-testing set-up
The participants were asked to wear shorts. They were positioned in the knee extension weight stack machine, and the axis of the device was aligned with the participant’s knee axis (dominant leg). The shin pad was kept as low on the tibia as possible, proximal to the lateral malleolus, with the participant still able to dorsiflex their ankle to a neutral position. Anthropometric measurements were taken in this position: thigh length (greater trochanter to lateral femoral condyle), leg length (knee joint line to the marked point for tibial pad), tibial pad location (mid-point on lateral malleolus to the tibia pad mark) and used to standardize the location of resistance on the distal tibia across the devices.
The participants performed 5 minutes of cycling on a stationary bike (75–100 watts), with no resistance, for warm-up before conducting measurements. As part of the warm-up, the participants performed one sub-maximal trial (50–75% of possible maximum effort) and 1–2 trial(s) with maximum intensity on each device. During the warm-up and practice trials, participants were instructed to build their force gradually and then hold the maximum effort. The instruction was “once the tester says start, build the force to reach the maximum effort in 1–2 seconds and hold the maximum until the tester says stop” (which was standardized to 5 seconds)”. No visual or verbal feedback was provided during warm-up or testing, and a 5-second contraction time was used to obtain a maximum force value [61]. A one-minute rest was provided between the sub-maximal and maximum trials. Then, three maximum isometric force (Fmax) trials were performed with a 1-minute rest between the trials to avoid fatigue [62]. A 5-minute rest period was given between the devices and 10 minutes between the raters (Fig 2).
Testing protocol using the HHD
After the participants and the HHD were set up for testing (Fig 3), the participants were instructed to keep their trunks straight and hold the sides of the plinth to avoid any backward movement of the trunk. HHD force application by the testers was at a right angle to the tibia. Before starting the measurement, the HHD force was reset to zero. A second investigator recorded the maximum force from the HHD screen, and the time-series data were recorded at 40Hz.
Photo of the set-up for measuring isometric knee extensor strength on the dominant side using a commercially available HHD (Lafayette Instruments, US, model no. - 01165). The participant was positioned in high sitting at the end of a treatment table with legs hanging off the side at 90⁰ of knee flexion and posterior thigh in complete contact with the treatment table. The proximal thigh was fixed to the treatment table by a mobilization belt. To ensure the angle of the knee joint was at 90⁰ before each trial, a universal goniometer was attached using Velcro with the axis at the lateral femoral condyle, fixed arm directed towards the greater trochanter, and movable arm towards the lateral malleolus. The HHD units were set to Newtons (N) and measurement time to 5 seconds before placing the device on the participant’s tibia. The HHD is strapped to the tester’s dominant hand so that the tester can place the measurement side of the HHD on the previously marked point of the participant’s tibia. The plinth height was raised to allow the tester to stabilize their elbow at 90 degrees of flexion against their body when providing resistance perpendicular to the tibia against the participant’s knee extensor force.
Testing protocol using the ND
Set-up of the ND.
Attaching and anchoring the ND to the weight stack requires aligning the device with the center of the weight stack and sliding the adjustable base under the frame of the weight machine. The device should be lifted so the adjustable base of the unit is in contact with the base of the weight machine frame prior to pushing the weight stack pin through the eyebolt of the ND. Gross adjustment to align the eyelet with the hole of the top plate on the weight stack is through a pin and slide mechanism. Fine adjustment is via a nut attached to the eyelet to adjust the height within fractions of an inch for the pretension of the load cell. The entire process takes less than 5 minutes. Further procedures to run the test require the patient being fitted into the machine to align the joint being tested to the axis of the machine lever arm, measuring the distance between the axis of rotation and the point of force development, and setting up the test information required. Test information includes auto-generated patient ID, test type and the number of trials, notation of the involved side, patient’s diagnosis, test limb, and joint angle. Other information that can be saved is turning on enabling of threshold and percentage of threshold and providing notes on set-up or other important information. Once saved, the Bluetooth is connected, and the test is ready to run.
Testing by using ND.
Once the participants and the ND were set (Fig 4), the load cell force output was set to zero. The force data were collected by the custom-built software at 10 Hz. The software was set to account for the machine’s CAM design at 90 degrees of knee flexion. The second investigator recorded the maximal force values, and time-series data were recorded.
Photo of the set-up testing knee extensor strength with the ND (left top panel) and the accompanying generated force-time curve (left bottom panel; X-axis is time, and Y-axis is the force in newtons). Participants were seated in the knee extension weight stack machine (Life Fitness, Illinois, USA; right panel). The axis of the participant’s knee joint (tip of lateral femoral condyle) and the machine axis were aligned. Participants were positioned so that the seat fully supported their thighs, and pillows were used when necessary to ensure a trunk-neutral position. The strapping, belt fixation, and goniometer placement procedures were the same as the HHD protocol. Knee flexion was aligned to 90⁰, and participants were asked to hold the side handles of the chair similar to the HHD protocol to maintain the neutral trunk alignment. The tibial pad was located at the mark on the tibia. The ND was inserted into the weight stack, leveled, and adjusted so no slack was present in the cable system.
Testing protocol using the IKD
The participants were asked to sit in the IKD machine chair (Cybex, Humac Norm, US; Fig 5). Force was recorded (kg), and the device was set to isometric mode with maximum force recorded and time-series data collected at 100 Hz. Fmax and time-series data were converted to Newtons (N) for analysis.
Photo of the set-up for testing knee extensor strength using the IKD. The participants were seated in the IKD machine chair (Cybex, Humac Norm, US), with the lateral epicondyle of the tested side femur centered with the IKD’s input arm axis by adjusting the chair’s position. The hip knee adapter stabilizing pad was kept against the participants’ tibia at the tibial marking by adjusting the knee/hip adapter and stabilizing pad until the bar was at the correct height. The investigators strapped the torso, thigh, and tibial harness to provide stability during the test. The knee angle (90⁰ of flexion) was manually adjusted using the software and confirmed with a universal goniometer.
Statistical analyses
Statistical analysis was conducted using the SPSS version 28 (Armonk, NY: IBM Corporation). Demographics (Mean, SD, N) were calculated for participants’ general characteristics. The normality of data was checked by the Shapiro-Wilk test. The average value (Mean, SD) of three maximum force trials for each test was used for analyses. Data were analyzed as all participants combined and separately by group (HV, OA). Session one data was utilized for within-day interrater reliability analyses and ND reliability (intrarater, second rater data) using a two-way mixed random absolute agreement model (ICC2,3). Data from both sessions was used for interrater between-day reliability of the HHD and ND using a two-way random model absolute agreement (ICC 2,3) [63]. ICC values were interpreted as follows: < 0.5; poor reliability, 0.5 to 0.75; moderate reliability, 0.75 to 0.90; good reliability, and > 0.90; excellent reliability [64]. The standard error of the mean assessed absolute reliability (SEM) = SDpooled × √ (1 –ICC), where SDpooled is the mean of both SDs. Minimal detectable difference (MDD) was calculated using the formula MDD = SEM × z score × √2, and the MDD 95% (z score = 1.96).
The data from the second testing session were used to analyze concurrent validity. To assess systematic bias, Bland-Altman plots, including the limits of agreement (LoA), were created between each pair of devices for all participants, and HV and OA groups separately in MATLAB (The MathWorks, Inc., Natick, MA, USA) [65]. One-way repeated measure ANOVA was implemented to compare the maximal force value across the three devices, and a pairwise comparison was performed to obtain the differences between each pair of devices. Pearson correlations (r) of maximum force values between each pair of devices were calculated and interpreted as negligible (0.0–0.10), weak (0.10–0.39), moderate (0.40–0.69), strong (0.70–0.89) or very strong correlation (0.90–1.00) [66]. Force-time curves representing force generation patterns between each pair of devices for each participant were down-sampled to 10 Hz, spline interpolated, and correlated in MATLAB. All statistical analyses were two-tailed, and the significance level (p) was < 0.05.
Results
Demographics and general characteristics
Twenty-nine HV (age: 28±7) and 15 OA (age: 38±13) participants (total n = 44) were deemed eligible and participated in this study. For the reliability analysis, six healthy individuals’ data were excluded from the study due to device errors, and therefore, 23 HV and 15 OA participants were considered for reliability analysis. All 29 HV and 15 OA participants’ data were used for validity testing. A detailed description of the participant demographics, anthropometric data, physical activity levels, and testing data pain intensity are presented in Table 1.
Reliability
Test-retest reliability of novel device.
The test-retest reliability of the ND (Table 2) ICC(2,3) was 0.97 (with narrow CIs) for all participants and did not change after group separation. The SEM was 53 N, and MDD95 was 148 N when all participants were included.
Within- and between-day interrater reliability.
Table 3 provides the within-day and between-day interrater reliability (ICC, CI), including the group mean Fmax values, SEM, and MDD95 of all participants and HV and OA groups separately.
Within-day interrater reliability.
The within-day interrater reliability of all participants was ICC2,3 = 0.88 for the HHD and ICC2,3 = 0.93 for the ND. In all analyses, the HHD had a wider ICC CI than the ND. Separate group analysis reveals that the reliability of the HHD was higher in the OA group compared to the HV group. In contrast, reliability of ND was not different between OA and HV groups. The results also showed that the HHD has a higher ICC value in the OA compared to the HV group. When all participants were considered, the measurement from ND had a greater SEM and MDD95 compared to HHD, principally due to the larger pooled between subject SD when using the ND. Similarly, in both the groups, the SEM and MDD95 values were lower for HHD than ND.
Between-day interrater reliability.
The interrater reliability values were good for both the HHD (ICC2,3 = 0.87) and ND (ICC2,3 = 0.89), across all participants. However, in the separate group analysis, both devices demonstrated lower reliability in the OA group, than the HV. In all analyses, the HHD had a wider ICC CI than the ND. Similar to the within-day interrater reliability, all participants had lower SEM and MDD95 values with the HHD than the ND.
Validity
One-way repeated measure ANOVA (Table 4) indicates a statistically significant difference in maximum force generated across the three devices (p < 0.001). Pairwise probability comparison demonstrates differences between the HHD and IKD (p < 0.001, MDabs = 58 N), while no statistically significant difference was found between the HHD vs. ND and ND vs. IKD. Pearson correlations reveal the highest correlation between ND vs. IKD (r = 0.81) compared to the correlation between HHD vs. ND (r = 0.72) and HHD vs. IKD (r = 0.74). Force-time curves (representing force generation patterns across time) for each participant (Fig 6) were correlated between each pair of devices and then averaged. We found the highest correlation average between the ND vs. IKD (0.96 ± 0.05).
Representative time-force curves, averaged across three trials, for two participants (time on the X-axis and force on the Y-axis). The force-time curve mean and standard deviation (shadow) of the ND (red) are similar to those of the IKD (purple). However, the HHD force-time curve (green) differs from those of ND and IKD.
Bland-Altman plots (Fig 7) indicate systematic bias of the HHD when compared with IKD and ND. The plots also display proportionate bias when comparing HHD with IKD and ND as the HHD progressively increases the underestimation of participant force starting around 450 N. Systematic bias was slight, and no proportionate bias was observed between ND and IKD. The LoA ranges were fairly large between all the devices due to the sample’s between-subject variability.
These represent the mean difference, p-value, limits of agreement (LoA), and bias between each pair of devices. The upper row plots include all participants, the middle row the HVs, and the bottom row knee OA participants. Each graph represents the mean difference at the center (solid line), the upper and lower limits of agreement, (dashed lines), and the dots represent the between device difference in Fmax for each participant.
Discussion
The overall results denote that the ND has excellent device reliability for measuring knee extensor isometric strength within a session using the same rater. The within-day and between-day interrater reliability of the HHD was good, whereas good to excellent reliability was detected for the ND. The findings indicate that the ND demonstrates higher concurrent validity against the gold standard (IKD), while a significant difference in force and proportional bias was observed between HHD and IKD.
Test-retest device reliability
To objectively measure and document the response to certain rehabilitation interventions such as exercises and manual therapy, muscle strength assessment using a HHD or IKD is common. For those clinics without an IKD, testing larger muscle groups presents a challenge for obtaining accurate and consistent results. Study findings consistently report that the removal of rater-generated resistance from the strength testing protocol improves within-day and between-day reliability [29, 45–47]. Our ND eliminates the need for tester resistance by utilizing equipment already found in most clinics. We inserted a tension load cell into a weight stack machine and took advantage of the machine’s adjustability, ease of patient stabilization, and ability to select joint positioning thus creating a hands-free strength testing unit. Our ND demonstrated excellent intrarater test-retest reliability (ICC2,3 > .90) across both HV and participants with OA.
Within-day reliability in healthy individuals
Most studies using HHD for knee strength testing report only within-day intrarater reliability. Studies of conventional HHD protocols (manual resistance) reported good to excellent within-session intrarater test-retest reliability (ICC = 0.70–0.92) for knee strength in healthy young adults [21, 27]. Researchers have approached the problem by using a belt to anchor the HHD or mounting it to a frame attached to a bench or chair and demonstrated that using their frame-mounted system resulted in good to excellent (ICC = 0.89–0.98) within-day interrater reliability [34, 45–47]. In a within-day test-retest study that mounted the HHD to PVC, they reported excellent intrarater (ICC = 0.93) reliability in 15 young, healthy individuals [39]. A study by Sung et al. modified their HHD protocol by anchoring the HHD to a frame but had the tester manually stabilize the pelvis during testing of knee extensor strength and reported excellent intrarater test-retest reliability (ICC = 0.98) [41]. The ND in our study demonstrated excellent (ICC2,3 > .90) within-day intrarater test-retest reliability in both HV and OA participants, and the narrow CIs suggest the ability of the device to capture reliable data even in participants with greater strength.
In clinics, multiple examiners often gather data for intervention progression decisions and documentation. Variability among clinical testers has been commonly reported in the literature. The potential for inconsistency among testers necessitates a device that minimizes sources of error to make it suitable for use in research and practice [67, 68]. Therefore, a majority of this study specifically focuses on interrater reliability to illustrate a worst-case scenario within the clinic. We found that the within-day interrater reliability of the HHD was good when the HV and OA participants’ data were analyzed together. However, the reliability of the HHD was lower in HV participants compared to knee OA, implying that the amount of force generated by the participants plays a crucial role in HHD reliability. Further, a wider CI was observed in the HV compared to OA individuals, again suggesting the testers’ strength could influence measurement interrater reliability. We found a study by Sung et al., that reported within-day interrater reliability with a HHD as excellent (ICC = 0.98) which they achieved by mounting the HHD to an external frame [41]. This suggests it is necessary to move beyond manual resistance to a system that augments participant and joint positioning and dynamometer stabilization.
Between-day reliability in healthy individuals
In clinical settings, knee extensor strength assessment is an essential indicator of response to rehabilitation and requires testing during different treatment sessions. Variability in quantitative measurements increases, and the agreement reduces when testing on separate days. Therefore, addressing the between-day interrater reliability contributes to the uniqueness and acceptability of devices. Only a few studies reported between-day reliability of conventional HHD testing and only in healthy individuals with studies reporting good to excellent interrater (ICC = 0.77–0.98) reliability between sessions [27, 32, 33, 69]. In parallel, the present study found good reliability using the HHD with manual resistance in HV, when protocol repeatability, participant stabilization, and tester positioning were optimized. On the other hand, the ND demonstrated excellent between-day interrater reliability in HV with a tight CI and good reliability in participants with OA. Comparing our SEM and MDD95 values to prior studies reporting between-day reliability is complicated by the tester differences and participant samples used. We found one paper reporting between-day interrater reliability for knee extensors using a dynamometer and strap and a few between-day intrarater studies reporting these values and only in healthy participants. Morin et al. reported a between-day interrater reliability of (ICC = 0.95) with an SEM 1.69% (2.47 Nm) and MDC95 4.69% (6.84 Nm), which represent a lower percent of the mean than our reported values [33]. Mentiplay et al., reported between days intrarater values ranging between 7.7 to 8.7% of the mean force for their SEM and 21.4 to 24.2% of the mean for the MDC95 using a HHD with manual resistance for testing knee extensor strength [32]. These values compare to the SEM of 6.8% and MDD95 of 19.1% in our study for interrater HHD. In studies where the HHD was fixed by a belt, a lower percentage of mean values were reported for the SEM 5.21% and MDC95 14.78% [38]. Martins et al. also performed a between-day intrarater study reporting an SEM of 12% and MDD95 of 34% for a HHD mounted to the arm of an IKD system [36]. This situation more closely matches our ND in which we found an SEM of 10.8% and MDD95 of 30.1%. The higher MDD95 in our study is partially driven by our HV large between-subject variance, which we attribute to the concurrent validity demonstrated by the ND. For comparison, we found one study that reported percent of mean force SEM and MDD95 values of 5.7% and 15.7%, respectively, when measuring knee extensor strength with an IKD [32].
Within and between-day reliability in a clinical population
Few studies assessing knee strength in a clinical population have been conducted that reported intra- and interrater reliability when using a HHD. Studies testing intrarater knee extensor strength within-day were located using participants with OA (ICC = 0.92–0.97) and hematological malignancy (ICC = 0.94) [40, 70]. Another study reported excellent knee extensor muscle strength reliability using a HHD within-day intrarater (ICC = 0.98) and interrater reliability (ICC = 0.95–0.96) in an unspecified rehabilitation sample [71]. Additional studies conducted in arthritic populations (e.g., TKR, THR, OA) modified their HHD protocols and reported that their redesigned system enhanced (ICC = 0.92–0.99) interrater reliability during a single session [35, 40]. The potential drawback of developing individual methods for improving HHD reliability of strength measurements in larger muscle groups, is the potential lack of consistency across raters and the time needed to set up the testing protocol within the clinic setting. The present study found good within or between-day interrater reliability using the HHD in a tightly controlled protocol, while the ND demonstrated excellent within-day or good between-day interrater reliability in participants with OA, with easier setup.
Concurrent validity
Most prior research identified that HHDs underestimate the force compared to the gold standard (IKD). A study in older adults found approximately 21 Nm and 27 Nm reduced torque by conventional HHD compared to IKD on the left and right knee extensors, respectively [69]. About a 15 Nm difference in knee extensor torque was found even after altering the participant’s position to supine lying with the knee in 35⁰ of flexion, which provided better leverage to the tester, offering resistance [42]. A few studies used belt-fixed HHD to minimize the role of the tester’s resistance and discerned that the HHD either underestimated or over-estimated the force compared to IKD [37, 43, 44]. Bohannon et al. reported a 35 Nm underestimation in knee extensor torque in HV [43]. Similarly, another study detected 230 N less knee extensor force obtained with the HHD in healthy individuals [44]. In contrast, Lesnak et al. showed belt stabilized HHD overestimates torque 19.4 Nm on average, and the difference was above the reported MDD [37]. However, the lack of adequate proximal segment stabilization owing to the possibility of synergistic muscle groups contributing to the recorded force production could account for overestimations. In addition, a higher possibility of dynamic movement is suspected during some modified and manually resisted HHD strength protocols, risking measurement consistency and validity. We observed significantly lower forces generated during HHD (58 N) testing compared with IKD, while no statistically significant difference in force was found between the ND and IKD (34 N). The differences in force generated by HHD and ND compared to IKD were below MDD95, and the ND more closely matched that of the gold standard. One of the primary reasons behind the higher SEM and MDD95 for the ND is the greater variability in participant generated force, particularly in the HV, as the ND does not appear to demonstrate a ceiling effect or proportional bias as noted in the HHD vs IKD Bland-Altman plots. The HHD increasingly underestimated the force generated by the participants starting around 450 N. This is a major limitation of the use of a HHD with manual resistance in clinical settings, especially when measuring the strength of large muscle groups and in participants with greater strength [43]. In contrast, the ND did not display proportionate bias, suggesting its validity for application in large muscle group testing and its capture of true isometric strength.
We found a strong correlation among maximum force values obtained by all the instruments; however, the highest correlation coefficient was demonstrated between the ND and IKD. Earlier studies of HV focusing on younger and older participants documented a strong correlation (0.78–0.92) with IKD either by using the conventional HHD technique or when placing the HHD on the tibial pad of a weight stack [32, 36, 42]. The possible reason behind a greater correlation between HHD and IKD in those studies, in contrast to our research, might be attributed to methodological differences such as manual resistance, unstandardized stabilization of the proximal segment, and or positioning of the participants. Another factor likely contributing to a lower correlation of the maximum force produced in our study is between participant variability in our data, as both HV and the clinical population were included in the validity analysis. A potential reason for the lower force generation and less agreement when using the HHD could be pain or discomfort reported at the tibial contact site with the HHD [34]. In our study, 34/44 individuals reported contact site pain during HHD testing, whereas only 19/44 and 11/44 participants reported tibia contact discomfort during testing with the ND and IKD, respectively.
Diverging from other currently available literature, we also assessed the force-time series correlation. We found the highest correlation between ND and IKD with a small SD which illustrates that the force generation pattern matched well with the IKD. This finding suggests that in those clinics without an IKD, the ND could be an efficient means for evaluating rate-of-force/torque development, which is a variable of interest in strength recovery metrics [72].
In fitness settings and clinics without IKD, but with weight stack machines, use of the ND can expand strength testing capabilities of larger muscle groups without significant increases in practitioner time or client burden. The typical testing time using the ND, inclusive of set up, 2–3 warm-up repetitions, and 3 test trials on each limb, is generally 15–20 minutes and allows for more consistent testing positions and easier stabilization of clients than using a HHD. The reduced ND setup time and less complicated software shorten the testing time by15-20 minutes in comparison to an equivalent IKD protocol while providing comparable reliability and validity. Practitioners should be sure to consistently test using the same weight stack machine when assessing patients over time, as different weight stack machines could produce slightly different results due to manufacture designs and specifications.
There were several limitations to this study. First, the major limitation of the ND compared to the IKD is that its application is limited to isometric testing. Second, our study is limited to the reliability and validity of knee extensor isometric strength. At the same time, we believe our findings will translate to testing other large muscles (e.g., hip extensors, hamstrings, hip abductors). However, our device was not tested on other muscle group specific weight stack machines. Third, given our participant directions to gradually build force during the test, we could not demonstrate our device’s ability to provide accurate rate force development data. However, given the high correlation (and tight SD) between the ND force-time curves with that of the IKD, we see no reason why force-time curves from the ND would demonstrate a vastly different rate of force development value from those of the IKD. Fourth, our testers were all fit males. Thus, the findings of the HHD reliability and concurrent validity may not translate to all clinicians or fitness professionals using the HHD as described in our study.
Conclusion
The study supports both within- and between-day interrater test-retest reliability, and maximum force and force-time curve validity of this ND and testing paradigm. The findings suggest that when testing knee extensor force production, the ND is comparable to an IKD and may be superior to HHD in healthy individuals and those diagnosed with mild to moderate knee OA. The HHD protocol used in this study resulted in a ceiling effect, unlike the ND and IKD, indicating that using an HHD to measure larger muscle groups may not fully represent a participant’s strength and is dependent on the resistance offered by the tester. Excellent device test-retest reliability and validity, portability, safety, and the hands-free protocol made possible by the ND suggest potential utility within physical therapy and fitness settings for strength testing large muscle groups such as knee extensors.
Acknowledgments
We want to acknowledge Trey Brindle, PhD, and Akil Piggott, DPT, who were integral in participant recruitment and data collection, and Sri Balasubramanian, PhD, a collaborator and fundamental part of the device development team.
References
- 1. Majewski M., Susanne H., and Klaus S., Epidemiology of athletic knee injuries: A 10-year study. Knee, 2006. 13(3): p. 184–8. pmid:16603363
- 2. Joseph A.M., et al., A multisport epidemiologic comparison of anterior cruciate ligament injuries in high school athletics. J Athl Train, 2013. 48(6): p. 810–7. pmid:24143905
- 3. Ward S.H., et al., Quadriceps Neuromuscular Function and Jump-Landing Sagittal-Plane Knee Biomechanics After Anterior Cruciate Ligament Reconstruction. J Athl Train, 2018. 53(2): p. 135–143. pmid:29350554
- 4. Segal N.A., et al., Effect of quadriceps strength and proprioception on risk for knee osteoarthritis. Med Sci Sports Exerc, 2010. 42(11): p. 2081–8. pmid:20351594
- 5. Larsen J.B., et al., Muscle strength and functional performance is markedly impaired at the recommended time point for sport return after anterior cruciate ligament reconstruction in recreational athletes. Hum Mov Sci, 2015. 39: p. 73–87. pmid:25461435
- 6. Karanikas K., Arampatzis A., and Brüggemann G.P., Motor task and muscle strength followed different adaptation patterns after anterior cruciate ligament reconstruction. Eur J Phys Rehabil Med, 2009. 45(1): p. 37–45. pmid:18987563
- 7. Kamiya K., et al., Quadriceps Strength as a Predictor of Mortality in Coronary Artery Disease. Am J Med, 2015. 128(11): p. 1212–9. pmid:26169888
- 8. Swallow E.B., et al., Quadriceps strength predicts mortality in patients with moderate to severe chronic obstructive pulmonary disease. Thorax, 2007. 62(2): p. 115–20. pmid:17090575
- 9. Chandler J.M., et al., Is lower extremity strength gain associated with improvement in physical performance and disability in frail, community-dwelling elders?. Arch Phys Med Rehabil 1998. 79: p. 24–30. pmid:9440412
- 10. Ringsberg K., et al., Is there a relationship between balance, gait performance and muscular strength in 75-year-old women? Age and Ageing, 1999. 28: p. 289–293. pmid:10475866
- 11. Davis J.W., et al., Strength, Physical Activity, and Body Mass Index: Relationship to Performance-Based Measures and Activities of Daily Li&g Among Older Japanese Women in Hawaii. Journal of Americal Geriatrics Society, 1998. 46: p. 274–279.
- 12. Wisloff U., Helgerud J., and Hoff J., Strength and endurance of elite soccer players. Medicine & Science in Sports & Exercise, 1998. 30(3): p. 462–467. pmid:9526895
- 13. Kraemer W.J., et al., Effect of resistance training on women’s strength/power and occupational performances. Med. Sci. Sports Exerc, 2001. 33: p. 1011–1025. pmid:11404668
- 14. Beaudart C., et al., Assessment of Muscle Function and Physical Performance in Daily Clinical Practice: A position paper endorsed by the European Society for Clinical and Economic Aspects of Osteoporosis, Osteoarthritis and Musculoskeletal Diseases (ESCEO). Calcif Tissue Int, 2019. 105(1): p. 1–14. pmid:30972475
- 15. Reilly T., Bangsbo J., and Franks A., Anthropometric and physiological predispositions for elite soccer. J. Sports Sci, 2000. 18: p. 669–683. pmid:11043893
- 16. Burns S.P. and Spanier D.E., Break-technique handheld dynamometry: relation between angular velocity and strength measurements. Arch Phys Med Rehabil, 2005. 86(7): p. 1420–6. pmid:16003675
- 17. Harbo T., Brincks J., and Andersen H., Maximal isokinetic and isometric muscle strength of major muscle groups related to age, body mass, height, and sex in 178 healthy subjects. Eur J Appl Physiol, 2012. 112(1): p. 267–75. pmid:21537927
- 18. Weininger S., Kapur K.C., and Pecht M., Exploring Medical Device Reliability and Its Relationship to Safety and Effectiveness. IEEE Transactions on Components and Packaging Technologies, 2010. 33(1): p. 240–245.
- 19. Bannigan K. and Watson R., Reliability and validity in a nutshell. J Clin Nurs, 2009. 18(23): p. 3237–43. pmid:19930083
- 20. Bohannon R.W., Hand-held dynamometry: Adoption 1900–2005. Percept Mot Skills, 2006. 103: p. 3–4. pmid:17037637
- 21. Wikholm J.B. and Bohannon R.W., Hand-held Dynamometer Measurements: Tester Strength Makes a Difference. The Journal of orthopaedic and sports physical therapy, 1991. 13(4): p. 191–198. pmid:18796845
- 22. Brinkmann J.R., Comparison of a hand-held and fixed dynamometer in measuring strength of patients with neuromuscular disease. Orthop Sports Phys Ther 1994. 19: p. 100–104. pmid:8148862
- 23. Bohannon R.W., Reference Values for Extremity Muscle Strength Obtained by Hand-Held Dynamometry From Adults Aged 20 to 79 Years. Arch Phys Med Rehabil, 1997. 789: p. 26–32. pmid:9014953
- 24. Whiteley R., et al., Correlation of isokinetic and novel hand-held dynamometry measures of knee flexion and extension strength testing. J Sci Med Sport, 2012. 15(5): p. 444–50. pmid:22424705
- 25. Chamorro C., et al., Absolute Reliability and Concurrent Validity of Hand Held Dynamometry and Isokinetic Dynamometry in the Hip, Knee and Ankle Joint: Systematic Review and Meta-analysis. Open Med (Wars), 2017. 12: p. 359–375. pmid:29071305
- 26. Lu T.W., et al., Enhancing the examiner’s resisting force improves the reliability of manual muscle strength measurements: comparison of a new device with hand-held dynamometry. J Rehabil Med, 2007. 39(9): p. 679–84. pmid:17999004
- 27. Kelln B.M., et al., Hand-Held Dynamometry: Reliability of Lower Extremity Muscle Testing in Healthy, Physically Active, Young Adults. Journal of Sport Rehabilitation, 2008. 17: p. 160–170.
- 28. de Araujo Ribeiro Alvares J.B, et al., Inter-machine reliability of the Biodex and Cybex isokinetic dynamometers for knee flexor/extensor isometric, concentric and eccentric tests. Phys Ther Sport, 2015. 16(1): p. 59–65. pmid:24913915
- 29. Toonstra J. and Mattacola C.G., Test-retest reliability and validity of isometric knee-flexion and -extension measurement using 3 methods of assessing muscle strength. J Sport Rehabil, 2013. 22(1). pmid:22951307
- 30.
Mendoza M. and Miller R.G., Muscle Strength, Assessment of, in Encyclopedia of the Neurological Sciences (Second Edition), Aminoff M.J. and Daroff R.B., Editors. 2014, Academic Press: Oxford. p. 190–193.
- 31. Mavroidis C., et al., Smart portable rehabilitation devices. J Neuroeng Rehabil, 2005. 2: p. 18. pmid:16011801
- 32. Mentiplay B.F., et al., Assessment of Lower Limb Muscle Strength and Power Using Hand-Held and Fixed Dynamometry: A Reliability and Validity Study. PLoS One, 2015. 10(10): p. e0140822. pmid:26509265
- 33. Morin M., et al., Psychometric properties of a standardized protocol of muscle strength assessment by hand-held dynamometry in healthy adults: a reliability study. BMC Musculoskelet Disord, 2023. 24(1): p. 294. pmid:37060020
- 34. Hansen E.M., et al., Hand-held Dynamometer Positioning Impacts Discomfort During Quadriceps Strength Testing: A Validity and Reliability Study. Int J Sports Phys Ther, 2015. 10(1): p. 62–68. pmid:25709864
- 35. Gagnon D., et al., Reliability and validity of static knee strength measurements obtained with a chair-fixed dynamometer in subjects with hip or knee arthroplasty. Arch Phys Med Rehabil, 2005. 86(10): p. 1998–2008. pmid:16213245
- 36. Martins J., et al., Reliability and Validity of the Belt-Stabilized Handheld Dynamometer in Hip- and Knee-Strength Tests. J Athl Train, 2017. 52(9): p. 809–819. pmid:28787180
- 37. Lesnak J., et al., Validity of Hand-Held Dynamometry in Measuring Quadriceps Strength and Rate of Torque Development. International Journal of Sports Physical Therapy, 2019. 14(2): p. 180–187. pmid:30997270
- 38. Lipovšek T., Kacin A., and Puh U., Reliability and validity of hand-held dynamometry for assessing lower limb muscle strength. Isokinetics and Exercise Science, 2022. 30(3): p. 231–240.
- 39. Jackson S.M., et al., Intrarater reliability of hand held dynamometry in measuring lower extremity isometric strength using a portable stabilization device. Musculoskelet Sci Pract, 2017. 27: p. 137–141. pmid:27476066
- 40. Koblbauer I.F., et al., Reliability of maximal isometric knee strength testing with modified hand-held dynamometry in patients awaiting total knee arthroplasty: useful in research and individual patient settings? A reliability study. BMC Musculoskelet Disord, 2011. 12: p. 249. pmid:22040119
- 41. Sung K.S., Yi Y.G., and Shin H.I., Reliability and validity of knee extensor strength measurements using a portable dynamometer anchoring system in a supine position. BMC Musculoskelet Disord, 2019. 20(1): p. 320. pmid:31286912
- 42. Martin H.J., et al., Is hand-held dynamometry useful for the measurement of quadriceps strength in older people? A comparison with the gold standard Bodex dynamometry. Gerontology, 2006. 52(3): p. 154–9. pmid:16645295
- 43. Bohannon R.W., et al., Adequacy of belt-stabilized testing of knee extension strength. Journal of strength and conditioning research, 2011. 25(7): p. 1963–1967. pmid:21399535
- 44. Katoh M. and Yamasaki H., Test-retest reliability of isometric leg muscle strength measurements made using a hand-held dynamometer restrained by a belt: comparisons during and between sessions. J Phys Ther Sci, 2009(21): p. 239–43.
- 45. Douma K.W., et al., Reliability of the Q Force; a mobile instrument for measuring isometric quadriceps muscle strength. BMC Sports Sci Med Rehabil, 2016. 8: p. 4. pmid:26900478
- 46. Verkerke G.J., et al., Precision, comfort and mechanical performance of the Quadriso-tester, a quadriceps force measuring device. 2003. 41(3): p. 283–289. pmid:12803292
- 47. Hartog J., et al., A portable isometric knee extensor strength testing device: test-retest reliability and minimal detectable change scores of the Q-Force ӀӀ in healthy adults. BMC Musculoskelet Disord, 2021. 22(1): p. 966.
- 48. O’Brien M., et al., A novel device to assess hip strength: Concurrent validity and normative values in male athletes. Phys Ther Sport, 2019. 35: p. 63–68. pmid:30471548
- 49. Bazett-Jones D.M. and Squier K., Measurement properties of hip strength measured by handheld dynamometry: Reliability and validity across the range of motion. Phys Ther Sport, 2020. 42: p. 100–106. pmid:31958744
- 50. Dunne C., et al., Validity, reliability, minimal detectable change, and methodological considerations for HHD and portable fixed frame isometric hip and groin strength testing: A comparison of unilateral and bilateral testing methods. Phys Ther Sport, 2022. 57: p. 46–52. pmid:35921781
- 51. Stark T., et al., Hand-held dynamometry correlation with the gold standard isokinetic dynamometry: a systematic review. PM R, 2011. 3(5): p. 472–9. pmid:21570036
- 52. Schrama P.P., et al., Intraexaminer reliability of hand-held dynamometry in the upper extremity: a systematic review. Arch Phys Med Rehabil, 2014. 95(12): p. 2444–69. pmid:24909587
- 53. Walter S.D., Eliasziw M., and Donner A., Sample size and optimal designs for reliability studies. Statistics in Medicine, 1998. 17(1): p. 101–110. pmid:9463853
- 54. Borg D.N., et al., Calculating sample size for reliability studies. PM R, 2022. 14(8): p. 1018–1025. pmid:35596122
- 55. Saltin B. and Grimby G., Physiological analysis of middle-aged and old former athletes Comparison with still active athletes of the same ages. Circulation, 1968. 38(6): p. 1104–1115.
- 56. Lindwall M., et al., The relationships of change in physical activity with change in depression, anxiety, and burnout: a longitudinal study of Swedish healthcare workers. Health Psychol, 2014. 33(11): p. 1309–18. pmid:24245832
- 57. Rodjer L., et al., Self-reported leisure time physical activity: a useful assessment tool in everyday health care. BMC Public Health, 2012(12): p. 693. pmid:22920914
- 58. Apullan F.J., et al., Usefulness of self-reported leisure-time physical activity to predict long-term survival in patients with coronary heart disease. Am J Cardiol, 2008. 102(4): p. 375–9. pmid:18678290
- 59. Aires N., Selmer R., and Thelle D., The validity of self-reported leisure time physical activity, and its relationship to serum cholesterol, blood pressure and body mass index. A population based study of 332,182 men and women aged 40–42 years. Eur J Epidemiol, 2003. 18(6): p. 479–485. pmid:12908712
- 60. Roach K.E., et al., Test‐retest reliability of patient reports of low back pain. The Journal of Orthopaedic and Sports Physical Therapy, 1997. 26: p. 253–259. pmid:9353688
- 61. Moudgil R. and Karpovich P.V., Duration of a Maximal Isometric Muscular Contraction. Research Quarterly. American Association for Health, Physical Education and Recreation, 2013. 40(3): p. 536–539.
- 62.
Astrand P.O. and Rodahl K., A text book of work physiology: Physiological bases of exercise. 2nd Edition ed. 1977, New York: McGraw-Hill Book Company.
- 63. Weir J.P., Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. Journal of strength and conditioning research, 2005. 19(1): p. 231–240. pmid:15705040
- 64. Koo T.K. and Li M.Y., A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med, 2016. 15(2): p. 155–63. pmid:27330520
- 65. Bland J.M. and Altman D.G., Statistical methods for assessing agreement between two methods of clinical measurement. Lancet, 1986. 1(8476): p. 307–10. pmid:2868172
- 66. Schober P., Boer C., and Schwarte L.A., Correlation Coefficients: Appropriate Use and Interpretation. Anesth Analg, 2018. 126(5): p. 1763–1768. pmid:29481436
- 67. McHugh M.L., Interrater reliability: the kappa statistic. Biochem Med (Zagreb), 2012. 22(3): p. 276–282. pmid:23092060
- 68. Tuijn S., et al., Reducing interrater variability and improving health care: a meta-analytical review. J Eval Clin Pract, 2012. 18(4): p. 887–95. pmid:21726359
- 69. Arnold C.M., et al., The reliability and validity of handheld dynamometry for the measurement of lower-extremity muscle strength in older adults. J Strength Cond Res, 2010. 24(3): p. 815–824. pmid:19661831
- 70. Knols R.H., et al., Hand-held dynamometry in patients with haematological malignancies: measurement error in the clinical assessment of knee extension strength. BMC Musculoskelet Disord, 2009. 10: p. 31. pmid:19272149
- 71. Pinto-Ramos J., et al., Handheld dynamometer reliability to measure knee extension strength in rehabilitation patients-A cross-sectional study. PLoS One, 2022. 17(5): p. e0268254. pmid:35580110
- 72. Angelozzi M., et al., Rate of force development as an adjunctive outcome measure for return-to-sport decisions after anterior cruciate ligament reconstruction. J Orthop Sports Phys Ther, 2012. 42(9): p. 772–80. pmid:22814219