https://orcid.org/0000-0003-4049-9291
Figures
Abstract
Quantitative running gait analysis is an important tool that provides beneficial outcomes to injury risk/recovery or performance assessment. Wearable devices have allowed running gait to be evaluated in any environment (i.e., laboratory or real-world settings), yet there are a plethora of different grades of devices (i.e., research-grade, commercial, or novel multi-modal) available with little information to make informed decisions on selection. This paper outlines a protocol that will examine different grades of wearables for running gait analysis in healthy individuals. Specifically, this pilot study will: 1) examine analytical validity and reliability of wearables (research-grade, commercial, high-end multimodal) within a controlled laboratory setting; 2) examine analytical validation of different grades of wearables in a real-world setting, and 3) explore clinical validation and usability of wearables for running gait analysis (e.g., injury history (previously injured, never injured), performance level (novice, elite) and relationship to meaningful outcomes). The different grades of wearable include: (1) A research-grade device, the Ax6 consists of a configurable tri-axial accelerometer and tri-axial gyroscope with variable sampling capabilities; (2) attainable (low-grade) commercial with proprietary software, the DorsaVi ViMove2 consisting of two, non-configurable IMUs modules, with a fixed sampling rate and (3) novel multimodal high-end system, the DANU Sports System that is a pair of textile socks, that contain silicone based capacitive pressure sensors, and configurable IMU modules with variable sampling rates.
Clinical trial registration: Trial registration: NCT05277181.
Citation: Mason R, Godfrey A, Barry G, Stuart S (2023) Wearables for running gait analysis: A study protocol. PLoS ONE 18(9): e0291289. https://doi.org/10.1371/journal.pone.0291289
Editor: Daniel Ramskov, Professionshøjskolen UCN: Professionshojskolen UCN, DENMARK
Received: February 23, 2023; Accepted: August 21, 2023; Published: September 11, 2023
Copyright: © 2023 Mason et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: No datasets were generated or analyzed during the current study. All relevant data from this study will be made available upon study completion. The study complies with the General Data Protection Regulation (GDPR) and Data Protection Act 2018, which require data to be de-identified as soon as it is practical to do so. All data samples collected as part of this study are anonymized with participants being assigned a unique study code. Any information pertaining to personal details is kept in locked filing cabinets in the biomechanics laboratory Northumbria University and is only available to research staff directly running the study. All data are entered into an electronic database using unique study codes for each participant and are securely stored on a password-protected computer database. Any significant protocol modifications during this study will be communicated to the trial registry. Deidentified research data will be made publicly available (online repository) when the study is completed and published. Results will be disseminated through academic outputs (e.g., national and international conferences and publications).
Funding: This work was funded by a PhD studentship from Northumbria University in collaboration with DANU Sports Ltd. (Award number #120162) (PI: Stuart). This work was also supported by Parkinson’s Foundation grants (PF-FBS-1898, PF-CRA-2073) (PI: Stuart) and Private Physiotherapy Educational Foundation (PPEF - #368) (PI: Stuart). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: Rachel Mason is on a PhD programme that has received support from DANU Sports Ltd., with this potential conflict reviewed and managed by Northumbria University. All other authors declare no conflicts.
Introduction
Running is one of the most popular sport and recreational activities worldwide, as well as being a core component to many sports [1]. With the growing popularity of running, a concurrent increase in the incidence of running related injuries has also been reported [2, 3]. It is well established that an abnormal running gait is a contributing factor to running related injuries [4]. Quantitative running gait analysis, as a (clinical) tool for minimising injury risk and as a performance measure, has been well documented in the literature [5–7], with improvements in running gait, leading to maximized mechanical efficiency thus enhancing performance, and reducing injury risk [8].
A variety of methods exist to examine running gait, in low-resource settings practitioners have largely been limited to the use of subjective clinical observations using rating scales (e.g., The High-Level Mobility and Assessment tool), and/or 2D video analysis, which may not be sensitive to subtle changes in performance with training or injury [9–11]. To analyse less readily identifiable outcomes, such as fine motor movement in spatiotemporal characteristics (e.g., stride length, stride time, cadence, speed), kinematic (e.g., angular velocity and joint angles) and kinetic (e.g., ground reaction forces, GRF), more cumbersome and expensive traditional (reference/gold-standard) gait laboratory equipment are required (e.g., 3D motion capture, force plate equipment, instrumented treadmills). Equally, use of laboratories for running gait assessment is further limited due to the need for trained practitioners to collect and analyse data and requirement to attend a bespoke setting. Furthermore, traditional lab-based approaches provide limited understanding of running from ‘real-world’ environments [12–14] and often use constrained protocols that may not represent usual running behaviour, such as unnatural force platform targeting [15]. Numerous studies have sought to overcome this issue by using instrumented treadmills, however, further studies demonstrate inconsistencies in running gait between over-ground and treadmill running [16]. In order to enhance understanding of running gait, further research in a natural running environment is required [15].
Wearable devices offer an alternative to overcome traditional assessment limitations and are becoming increasingly popular amongst runners, coaches and clinicians [17]. Wearables using accelerometers, gyroscopes, and magnetometers, applied individually or in combination as an inertial measurement unit (IMU), and ‘pressure-sensitive’ insoles enable examination of a combination of spatiotemporal, kinetic, and kinematic variables and have become a viable alternative due to their portability and affordability [18]. Wearables can quantify running gait outcomes in any setting (i.e., laboratory, outdoor/real-world), and have the potential to be an effective tool to capture the full duration and nuances of a run, which ultimately may enhance understanding of running performance, fatigue, and injury mechanisms [19, 20].
Wearables have seen an expansive increase in personal use, as well as application in industry and research. However, there is a lack of evidence for validity, reliability, and application of wearables for running gait assessment [19–22], with different grades of devices receiving different levels of testing. For example, many commercial devices receive little / no validation testing before commercial launch, whereas research-grade devices that are developed by academic researchers tend to receive validation testing but may not be applied within relevant populations to inform clinical or performance outcomes [18–22]. Therefore, this pilot study aims to examine the validity, reliability, and application of various grades of wearable devices (research-grade, commercially available and novel multi-modal) for running gait assessment. The presented protocol will follow an evidence-based approach to drive the appropriate adoption of fit-for-purpose wearable technologies [23]. An evaluation of the usefulness and utility is only applicable after gaining evidence and assurance that the underlying data obtain valid and reliable results to answer a given question [23].
Analytical validation aims
- Laboratory validation: Determine the concurrent validity and test re-test reliability of wearables (research-grade, commercial and novel multimodal devices) for running gait analysis against gold-standard reference measures in adult runners.
- Real-world validation: Investigate concurrent validity of real-world running gait as measured by different wearables (research-grade, commercial and novel multimodal devices) in adult runners.
Exploratory clinical validation aims
- Explore differences in running gait outcomes within different clinical (injured vs non-injured) and performance (elite vs novice) groups.
- Explore relationships between real-world running gait and demographic characteristics (e.g., sex, age) in different clinical and performance groups (e.g., those with a previous injury or different performance level).
- Investigate the usability of the wearable devices in adult runners.
Materials and methods
Study design
This is a cross-sectional study that will assess the validity and reliability of three grades of wearables (research, entry-level commercial and advanced / novel multimodal). The protocol was developed according to the Standard Protocol Items: Recommendations for Interventional Trials’ (SPIRIT) checklist [24], as appropriate (see S1 File). As this is a study protocol, no data has been included and conforms to PLOS data policy. The overall schedule and time commitment for trial participants are depicted in Fig 1 and S1 File.
The overall schedule and time commitment for study participants for both a) laboratory testing and b) real-world environment testing.
Ethical approval and trial registration
A Northumbria University research ethics committee granted ethical approval (Ref: 21603 and 33358) and this study conforms to the Declaration of Helsinki. The trial has been reviewed and registered with Clinicaltrials.gov (NCT05277181).
Participants
A total of 80 runners with a variety of training experience will be recruited and assessed within laboratory (n = 40) and real-world environments (n = 40). The common inclusion and exclusion criteria are outlined in Table 1.
Recruitment.
An advertisement poster and email text invitation will be sent via email to Northumbria University running clubs and running clubs within England. Recipients will be asked to pass on the recruitment poster to potential interested parties (i.e., family or friends). Recruitment began in January 2022. Those interested will then be given a participant information sheet and a letter concerning the study and consent. Written, informed consent will be collected from all participants.
Equipment
Reference standard: 3D motion analysis.
A 14-camera motion analysis system (VICON, Oxford, UK), sampling at 250Hz will be used. Calibration of the Vicon system will be conducted before each data collection. Sixteen reflective markers will be placed on the participants lower limb before testing, and a static calibration trial will be initially collected to form a musculoskeletal model [25]. Using a small amount of double-sided tape, the place of markers will be attached bilaterally to the following landmarks: anterior superior iliac spine, posterior superior iliac spine, mid-lateral thigh, lateral knee joint line, lateral mid-shank, lateral malleoli, calcaneal tuberosity, and base of the second metatarsal.
The participants’ specific information of weight, height, ankle width, knee width, and leg length (from posterior iliac spine to medial malleolus) will be measured in the lower body model [26]. The Plug-in-Gait (PiG) lower body model will be used to analyse movement at the joints and evaluate all parameters. The lower body will be modelled as seven segments (one pelvis, two thighs, two shanks, and two feet). A normal gait cycle will be defined from the initial heel-to-heel contact with the same limb. Additional information of the PiG calculations can be found on Vicon’s website. Fig 2 shows the participants’ setup of the anterior, lateral, and posterior views with the markers in place.
The positions for the three grades of wearables and reflective markers for 3D motion capture are shown.
Reference standard: Force plate system.
Two staggered 0.5-m-long force plates (AMTI, Watertown, MA, USA), sampling at 1000 Hz will be embedded in the middle of a 15m walkway.
Wearables.
With many wearables available, it is difficult to choose the most suitable [21]. Accordingly, this protocol includes options from three categories to suit varying user needs: (1) modifiable research-grade; (2) attainable (low-grade) commercial with proprietary software; and (3) novel multimodal high-end system [18]. Accordingly, the three wearable types presented here are: research grade AX6 (Axivity, https://axivity.com/), entry-level commercial DorsaVi ViMove2 (DorsaVi, www.dorsavi.com/vimove/) and advanced multimodal DANU System (DANU Sports, www.danusports.com/).
Typically, research-grade devices require users to have specialist / technical knowledge due to the tendency of devices to provide the raw data only. Therefore, the user often needs to create and implement algorithms to achieve tangible outcomes. However, these devices offer the flexibility to define wear location and creation of novel outcomes [27]. Consumer level / commercial wearables with proprietary software are sought to be user-friendly, requiring little technical knowledge or experience, producing outcomes automatically. For these devices, it is often unclear how reference limits are set, or established for such functions [27]. Rather than showing the raw data collected by the wearable, users typically export information after it has been analysed and filtered through an algorithm. More advanced multi-modal devices (e.g., combined IMU and pressure sensors in a single system) offer numerous sensing capabilities with multiple algorithms that are enabling laboratory grade sensing in any location. Accordingly, the integrated sensor arrangement enables a comprehensive assessment of running gait (i.e., spatiotemporal, and kinetic outcomes) and attempt to offer the advantages of both research-grade and commercial wearables.
Axivity AX6. The AX6 (2.3 × 3.3 × 0.8 cm, 11g) consists of a configurable tri-axial accelerometer (±2, ±4, ±8, ±16g) and tri-axial gyroscope (±125 –±2000°/s) with variable sampling capabilities (12.5Hz - 1600Hz), set via a configuration proprietary software (OmGUI). Here, the device will be configured to a sampling rate of 200Hz, ±16g accelerometer and ±2000°/s gyroscope. Although it can be worn on any anatomical landmark, due to lack of proprietary analytical software, it will be secured onto the talus joint of each shoe with medical tape, Fig 3. The AX6 is considered a valid technology which can capture data in controlled and free-living environments [28]. Data are recorded and stored locally on the device and download via OmGUI upon completion of recording.
DorsaVi ViMove2. The DorsaVi ViMove2 consists of two IMUs modules, each comprising a tri-axial accelerometer (±16g) with tri-axial magnetometer and tri-axial gyroscope (±250°/s), sampling at 100Hz. Modules will be placed onto the anterior surface of the mid-shank of the right and left tibia, as per manufacturer’s guidelines (Fig 4) [29]. Tibia sensor placement is based on the anthropometric measurement of height and a ruler is used to measure exact placement, which is the midpoint between the knee and ankle along the anterior surface of the tibia [30]. After placement of bilateral tibia sensors, the DorsaVi ViMove2 is calibrated via its ViPerform software. Data are transmitted through wireless channel to a recording and feedback device, from which data can be offloaded onto PC for further analysis.
DANU sports system. The DANU Sports System (Fig 5) is a pair of textile socks, to be worn on both feet. Each sock contains 15 silicone based capacitive pressure sensors, and an IMU module attached onto the anterior surface of the mid-shank of tibia. Each IMU module is Bluetooth enabled for data transmission and is comprised of two configurable tri-axial accelerometers (Accelerometer 1 ±2g, ±16g, and ±32g, Accelerometer 2 ±50g, ±100g, and ±200g), gyroscope (±2000°/s), magnetometer, and with variable sampling rates (60 – 250Hz). The IMU module includes in-built memory for data collection. For this protocol, the DANU Sports System will be configured to a default sampling rate of 250Hz, ±16g, ±50g accelerometers and ±2000°/s gyroscope. Data is collected via Bluetooth on Apple devices 2018 or later (iPad or iPhone devices are required to have at least Bluetooth 5.0 connectivity), and data processing is run through a custom-made Apple application for real-time feedback and visualisation.
Data collection
Prior to data collection, all participants regardless of study arm (laboratory or real-world) will complete a questionnaire to provide information pertaining to their demographics, lifetime athletic injury history, medical history, sporting pursuits and running personal bests (S1 Appendix). Injury will be classified as “any muscle, bone, tendon or ligament pain in the lower back/legs/knee/foot/ankle that caused a restriction or stoppage of running (distance, speed, duration or training) for at least 7 days or 3 consecutive scheduled training sessions, or that required the runner to consult a physician or other health professional” [31–33].
Laboratory testing.
Laboratory testing will be conducted at the biomechanics laboratory Northumbria University, Newcastle upon Tyne, United Kingdom. Within the controlled laboratory environment, we will conduct a concurrent data collection with the multimodal, commercial, and research-grade wearables to determine their validation and reliability for running gait analysis compared to laboratory reference standards (3D motion capture and force plates). Recruitment for the laboratory testing began in September 2021 with an expected end date September 2023.
Participants. Within the laboratory testing 40 individuals will be recruited, specifically 20 with and 20 without a history of previous lower limb injury (but fully recovered at time of assessment and able to run safely, in line with inclusion / exclusion criteria).
Laboratory reference set-up. As previously described, a 3D motion capture and force plate system will be used as the ‘gold-standard’ reference measures.
Running procedures. All participants will run overground and, on a treadmill, (Spirit fitness XT485). Participants will be provided with a standardised, neutral cushioning running shoe (Saucony Guide Runner) to wear during testing in order to remove bias [34]. Prior to testing, participants will run on the treadmill at a comfortable speed for a warm-up and to familiarise themselves with running on a treadmill. Participants will be allowed breaks at any point or permitted to abort the trial at any time. For the overground segment, participants will run at a comfortable self-selected speed overground for 10m intermittently, where they will be asked to foot-strike two staggered force plates in the middle of the run (~5m point). Practice trials will be performed prior to data collection to allow participants to adjust their start position to strike the force plates correctly. A total of five successful recordings for each of the left and right leg will be captured. For the treadmill segment, participants will be asked to run at five speeds, four of these speeds will be standardised (i.e., 8, 10, 12 and 14 km/hr) and one speed will be their self-selected speed. Self-selected speed will be determined by the participant’s 5km personal best. The order of speed will be consistent across participants, starting at the slowest speed and progressing to the fastest to ensure the safety of participants. Data will be collected for 60s at each speed.
Initial data processing. Data will be collected simultaneously by the measurement systems, at the commencement and conclusion of each task, the time will be synchronized with a laptop and recorded by the assessor (i.e., time-stamp extraction). At the end of each testing session, the wearable and reference device data will be download onto a computer and stored in secure servers for further data processing and analysis.
Reliability testing.
Each participant will complete the protocol in the same format in a repeated-measures design, approximately one week after the first session to establish the test-retest reliability.
Real-World environment testing.
Data from the novel multimodal, commercial, and research-grade wearables will be collected within real-world environments to test the clinical / performance validity of the wearables (i.e., can they differentiate or provide meaningful data on relevant populations). Recruitment for the real-world environment testing began in June 2023 with an expected end date October 2023.
Participants. Within the real-world environment testing 40 individuals, specifically novice/amateur (n = 20) and expert/sub-elite (n = 20) running performance level. Performance level will be based on their 5km running time (i.e., age graded performance % [35]).
Wearable location. Each participant will be equipped with the DANU Sports System (socks on both feet), two Axivity AX6 sensors attached to the shoelaces and two DorsaVi ViMove2 sensors on the tibia.
Running procedures. All participants, regardless of performance level, will be asked to complete a 5km run on a standardised route in North-East England. The course will be run on a mixture of trail paths and concrete paths, with 174ft elevation gain throughout. Information about conditions (e.g., environment and shoes) and objective load data (e.g., time and pace) will be collected. Participants will wear their own running shoes.
System usability scale.
Upon immediate completion of the real-world environment testing, participants will be asked to complete the System Usability Scale (SUS) to assess the usability of each of the wearables. The SUS is a simple scale based, multiple-choice questionnaire proposed by John Brooke in 1986 and has been widely adopted in product usability evaluations [36–38]. A final score is provided with interpretation based on a well-established reference standard, with an excellent reliability (0.85) [39]. Overall, the SUS is a quick and simple method for usability evaluation.
Outcome measures
The primary outcome measures for this study are running gait outcomes as measured by the wearables and are described in Table 2. A secondary outcome measures is usability. Usability refers to the effectiveness, efficiency, and user satisfaction rating of a product (various grades of wearables) in an particular environment by a user for a specific purpose (running gait analysis) [40].
Sample size calculation
This is the first study (pilot study) to use the novel multi-modal wearable system (DANU system) for gait analysis in any population, alongside other wearables and gold-standard measures. As such, there are few specific previous examples were available to guide estimates for sample size. Previous research that has shown statistically significant outcomes when investigating wearables for running gait assessment in sport have had sample sizes ranging from 1 [41] to 187 [42], with the average number of participants at 26 [21]. Thus, a sample size of 40 participants for analytical validation (i.e., comparison to gold-standards) is feasible and adequately powered [43, 44].
Regarding the exploratory clinical validation analysis, a sample size of 20 participants per group for the performance and clinical group analysis is adequate in this initial exploratory pilot study. While there are no specific rules for sample size calculations in pilot / initial exploratory studies, evidence suggests that 10–12 participants per group may be adequate, and that 20–30 participants per group would be enough to provide valid evidence to draw scientific conclusions [43–46]. Further to this, evidence suggests that to capture 85–90% of use issues 15–20 people per group are required for usability studies [46], therefore our sample of 20 participants per group should be adequate for this initial analysis. Following from the present pilot study, future research can calculate more specific sample sizes and control for relevant variables based on the presented findings.
Data processing
All collected data will be anonymised, stored, and saved in the main computer, password protected. Data processing will be performed in Vicon Nexus. All markers will be labelled, and marker trajectories were filtered using a fourth order low-pass Butterworth filter via dynamic plug-in gait model with 6 Hz cut-off frequency. Gait identification will be achieved through visual inspection of foot strike and toe off for consecutive strides over the 60s. A threshold of 20 N in ground reaction force will used to determine initial contact and toe-off. Kinetic variables will be normalized using base factors of gravitational constant g (9.81 m/s2), leg length, L (m), and body mass, M (kg).
Wearable device data (DANU Sport and DorsaVi ViMove2) will be processed as per the manufacturer’s guidelines, whereas the Axivity AX6 data processing will be carried out from the nine axis (accelerometer, magnetometer, and gyroscope) raw signal by a previously validated in-house algorithm [47]. A master data table will be created with all available outcomes, categorised by subject and date, including a column to indicate distinguishing factors (e.g., those with previous injury history, different performance levels).
Statistical analysis
Analysis common to all studies.
The statistical analysis will be performed using statistical software SPSS 25 (SPSS Inc., Chicago, Illinois, USA). Demographic characteristics will be summarised using descriptive statistics, including mean, standard deviation, median, minimum, maximum, and inter-quartile range for continuous or ordinal data and percentages for categorical data. The descriptive statistics will be tabulated and presented graphically for clarity. All data will be checked for normality distributed with Shapiro-Wilks tests before conducting parametric tests. Non-normal continuous distributions will be transformed where appropriate to meet the requirements of parametric tests; otherwise, the equivalent non-parametric tests will be adopted. Data will also be assessed graphically (such as histograms or scatter plots) for clarity of information. Statistical significance will be determined at p < 0.05 unless otherwise stated. Due to the experimental nature and novelty of the work and wearable systems, we will not control for multiple comparisons in order to avoid a Type II error.
Analytical validation analysis. To examine analytical (concurrent) validation of the wearable devices (research-grade, commercial, novel multimodal) for running gait measurement, we will compare outcomes to relevant reference measures (i.e., force plates for ground contact time) in all laboratory testing participants (n = 40). To compare running gait outcomes from the wearables and reference systems, the absolute difference, Pearson’s correlation (r), intraclass correlations (ICC), the level of agreement through the Bland Altman plots, and 95% Limits of agreement (LoA) will be calculated [48]. Accuracy will be evaluated in terms of root mean squared errors (RMSEs).
To determine test re-test reliability of the wearables, r, and ICCs between the two testing time-points will be calculated [49]. A predefined ICC performance scale will be used, defined as poor (<0.50), moderate (0.50–0.75), good (0.75–0.90) or excellent (>0.90) [50]. ICC is a common statistic for evaluating repeatability and is needed to calculate the minimally detectable change (MDC) [51, 52]. The standard error of measurement (SEM) and standard deviation (SD) will also be calculated. Laboratory based analytical validity and reliability outcomes will be descriptively reported and discussed across research-grade, commercial and novel multi-modal wearable devices.
To examine the analytical (concurrent) validation of running gait outcomes measured in real-world settings from the various grades of wearables in all real-world testing participants (n = 40), we will compare outcomes across the wearable devices (research, commercial and novel multi-modal) through widely used measures of agreement. The absolute difference, r, ICC, the level of agreement through the Bland Altman plots, 95% LoA, RMSEs will be calculated to evaluate agreement between outcomes generated by each wearable [48].
Exploratory clinical validation analysis. To provide an initial exploration of clinical (performance) validation, we will investigate group difference and relationships with clinically (or performance) relevant measures. For group differences (e.g., previous injury history or performance level), analysis of co-variance (ANCOVA), will be conducted to assess for differences between groups (injured vs non-injured, or elite vs novice) in the individual running gait outcomes (e.g., ground contact time etc.) from the novel multimodal, commercial and research-grade wearable devices during the running tasks, while controlling for demographics that are significantly related to the running gait outcomes. Relationships between running gait outcomes and demographic variables (e.g., sex, age, height, weight etc.) within groups (injured vs non-injured, or elite vs novice) will be completed through Pearson correlations within each grade of wearable device (research, commercial, multimodal).
To examine the usability of the wearable devices the SUS scores for each wearable will be calculated and visualised. To investigate differences in participant perception of the usability for each grade of wearable, separate analysis of variance (ANOVA) tests will be performed. The ANOVAs will determine the differences between the SUS scores within the different device grades between participant groups (e.g., injury history, performance level). The relationships between the SUS scores, demographics (e.g., sex, age, age-grade performance), and wearables characteristics (e.g., price, weight, and battery life) will be explored using Pearson correlations [39]. The strength of the correlation will be assessed based on Cohen’s criteria: correlations < 0.30 are considered small, correlations between 0.30 and 0.50 will be considered medium, and correlations > 0.50 are considered strong [53].
Data management and availability.
The study complies with the General Data Protection Regulation (GDPR) and Data Protection Act 2018, which require data to be de-identified as soon as it is practical to do so. All data samples to be collected as part of this study will be anonymised with participants being assigned a unique study code. Any information pertaining to personal details will be kept in locked filing cabinets in the biomechanics laboratory Northumbria University and is only available to research staff directly running the study. All data will be entered into an electronic database using unique study codes for each participant and securely stored on a password-protected computer database. Any significant protocol modifications during this study will be communicated to the trial registry. Data will be available from the study, once conducted, upon reasonable request from the principle investigator (SS). Results will be disseminated through academic outputs (e.g., national, and international conferences and publications).
Discussion
There is a growing need for simple, valid, reliable, and objective methods to evaluate the running gait away from the laboratory within real-world environments, yet with the availability of such a wide range of research-grade, commercial and high-end multimodal wearables users may find it difficult to select the most suitable [21]. Therefore, through evaluating validity and reliability of various grades of wearables and demonstrating their efficacy and usability in applied studies, this protocol seeks to enhance the understanding of wearables for objective measurement of running gait.
Analytical validation
This pilot study protocol was developed in response to recently reviewed evidence and study recommendations for assessing gait in ecologically valid environments [54]. Prior to applying wearables to sporting contexts it is critical to examine the validity and reliability of such technology. This proposed protocol will assess the validity and reliability of different grades of wearables (research-grade, commercial, novel multimodal) for running gait assessment, examining outcomes against ‘gold-standard’ reference measures (3D motion capture). Research-grade wearable devices have been developed by academic researchers, but few have been used in applied studies to inform clinical or performance decisions [21]. Similarly, despite the widespread use, fewer than 10% of commercially available wearables are validated against an accepted ‘gold-standard’ [27]. Reliability studies of wearables for running gait are less established [21]. An important implication of this study is that it is expected to provide the first evidence on the accuracy and reliability of using a multi-modal wearable device (DANU Sports System) during running gait, which will help develop the standard of wearables. At present, most wearables are limited in their assessments to only examination of selective spatiotemporal or kinematic outcomes, for example step frequency, step length, tibial acceleration and ground contact time are commonly cited [21]. Despite numerous studies establishing that running biomechanics cannot be described based on a single parameter [55–58]. Therefore, the use of a multi-modal system (e.g., combination of IMU and pressure insoles, or accelerometer and pressure insole) as proposed in this protocol will address such limitations and allow for more comprehensive assessment of running gait.
Exploratory clinical validation
Following the examination of analytical validity and reliability the protocol will seek to examine runners in a more natural running environment to provide initial evidence on clinical (or performance) validity (i.e., do the analytically valid outcomes provide something meaningful). By moving beyond the laboratory to more natural running environments (i.e., indoor, or outdoor running tracks, or sports venues), we will be able to evaluate whether the different grades of wearables (research-grade, commercial, novel multimodal) acceptably measure and provide meaningful insight in the specified context of use [23]. This study will determine what grade of wearable may be necessary to provide meaningful data on real-world running. The use of sub-group analyses will elucidate outcomes that are most representative of an individual’s running gait, as well as reveal the interactions between confounding demographic factors, injury status and performance level aid performance enhancement and reduce injury risk / occurrence [59–61]. The proposed protocol will allow for examination of such factors and hopes to provide clarity for researchers and practitioners on the applicability of certain wearables in context.
Usability
Technological, design and aesthetical characteristics are key areas together that affect the adoption of wearable devices in sport [62–64]. Wearable devices such as the ones described in this paper are able to monitor the performance of the user in a cost effective, accurate and non-invasive manner, yet they are of little use if they lack practicality, comfort, and aesthetical pleasure. Hence, a necessary step is to investigate the opinions of the targeted end-users. This protocol will seek to for investigate the usability of wearables during running activity via administration of a questionnaire. With any new technology it is essential to understand user acceptance in order to predict behavioural intention to use the system in the future. Therefore, by understanding user acceptance of the three systems we can gain further insight into future usage.
Practical implications
Understanding running gait is crucial in injury prevention and for improving performance. Creating a protocol that considers the validity and reliability of wearables for running gait assessment, consequently, allows practitioners to examine numerous key areas (i.e., fatigue, speed, surface, injury) in ecologically valid settings and without disrupting an individual’s training regime. Furthermore, the proposed protocol partly addresses some additional limitations observed in previous studies, including small sample sizes, examination of only one outcome and lack of consideration for the user’s comfort. Through using various wearables in both the lab and real-world environments, the suitability of each device not only in regard to running gait outcomes obtained but also practicality, time and cost-effectiveness, and user comfort can be evaluated.
Conclusion
This exploratory pilot study will provide evidence for the validity and reliability of running gait as measured via wearables of various grades (research-grade, commercial, novel multimodal) in adult runners. In addition, it will provide initial evidence on the relationships between demographic factors, injury status, and performance level on objectively measured running gait outcomes from different grades of wearables.
Supporting information
S1 Checklist. SPIRIT-Outcomes 2022 checklist (for combined completion of SPIRIT 2013 and SPIRIT outcomes 2022 items)a.
https://doi.org/10.1371/journal.pone.0291289.s002
(PDF)
References
- 1.
J. J. Andersen. "The State of Running 2019." https://runrepeat.com/state-of-running (accessed 27th July, 2021).
- 2. Fokkema T. et al., "Reasons and predictors of discontinuation of running after a running program for novice runners," Journal of Science and Medicine in Sport, vol. 22, no. 1, pp. 106–111, 2019/01/01/ 2019, pmid:29934211
- 3. Linton L. and Valentin S., "Running with injury: A study of UK novice and recreational runners and factors associated with running related injury," (in eng), J Sci Med Sport, vol. 21, no. 12, pp. 1221–1225, Dec 2018, pmid:29853263
- 4. Fredette A., Roy J.-S., Perreault K., Dupuis F., Napier C., and Esculier J.-F., "The Association Between Running Injuries and Training Parameters: A Systematic Review," Journal of Athletic Training, vol. 57, no. 7, pp. 650–671, 2021, pmid:34478518
- 5. Wang L., Hu W., and Tan T., "Recent developments in human motion analysis," Pattern Recognition, vol. 36, no. 3, pp. 585–601, 2003/03/01/ 2003,
- 6. Chen K. Y., Janz K. F., Zhu W., and Brychta R. J., "Redefining the roles of sensors in objective physical activity monitoring," (in eng), Med Sci Sports Exerc, vol. 44, no. 1 Suppl 1, pp. S13–23, Jan 2012, pmid:22157770
- 7. Tao W., Liu T., Zheng R., and Feng H., "Gait analysis using wearable sensors," (in eng), Sensors (Basel), vol. 12, no. 2, pp. 2255–83, 2012, pmid:22438763
- 8. Doyle E., Doyle T. L. A., Bonacci J., and Fuller J. T., "The Effectiveness of Gait Retraining on Running Kinematics, Kinetics, Performance, Pain, and Injury in Distance Runners: A Systematic Review With Meta-analysis," (in eng), J Orthop Sports Phys Ther, vol. 52, no. 4, pp. 192–a5, Apr 2022, pmid:35128941
- 9. Dingenen B., Barton C., Janssen T., Benoit A., and Malliaras P., "Test-retest reliability of two-dimensional video analysis during running," (in eng), Phys Ther Sport, vol. 33, pp. 40–47, Sep 2018, pmid:30005426
- 10. Ferber R. and Macdonald S., "Running mechanics and gait analysis," 2014.
- 11. Pipkin A., Kotecki K., Hetzel S., and Heiderscheit B., "Reliability of a Qualitative Video Analysis for Running," (in eng), J Orthop Sports Phys Ther, vol. 46, no. 7, pp. 556–61, Jul 2016, pmid:27266886
- 12. Dugan S. A. and Bhat K. P., "Biomechanics and analysis of running gait," (in eng), Phys Med Rehabil Clin N Am, vol. 16, no. 3, pp. 603–21, Aug 2005, pmid:16005396
- 13. Higginson B. K., "Methods of running gait analysis," (in eng), Curr Sports Med Rep, vol. 8, no. 3, pp. 136–41, May-Jun 2009, pmid:19436169
- 14. Norris M., Anderson R., and Kenny I. C., "Method analysis of accelerometers and gyroscopes in running gait: A systematic review," Proceedings of the Institution of Mechanical Engineers Part P-Journal of Sports Engineering and Technology, vol. 228, no. 1, pp. 3–15, Mar 2014,
- 15. Mann R., Malisoux L., Urhausen A., Meijer K., and Theisen D., "Plantar pressure measurements and running-related injury: A systematic review of methods and possible associations," Gait & Posture, vol. 47, pp. 1–9, 2016/06/01/ 2016, pmid:27264395
- 16. Chambon N., Delattre N., Guéguen N., Berton E., and Rao G., "Shoe drop has opposite influence on running pattern when running overground or on a treadmill," (in eng), Eur J Appl Physiol, vol. 115, no. 5, pp. 911–8, May 2015, pmid:25501676
- 17. Willy R. W., "Innovations and pitfalls in the use of wearable devices in the prevention and rehabilitation of running related injuries," Physical Therapy in Sport, vol. 29, pp. 26–33, Jan 2018, pmid:29172095
- 18.
Stuart S. et al., "Chapter 15—Sports medicine: bespoke player management," in Digital Health, Godfrey A. and Stuart S. Eds.: Academic Press, 2021, pp. 231–251.
- 19. Reenalda J., Maartens E., Buurke J. H., and Gruber A. H., "Kinematics and shock attenuation during a prolonged run on the athletic track as measured with inertial magnetic measurement units," Gait and Posture, Article vol. 68, pp. 155–160, 2019, pmid:30481697
- 20. Reenalda J., Maartens E., Homan L., and Buurke J. H. J., "Continuous three dimensional analysis of running mechanics during a marathon by means of inertial magnetic measurement units to objectify changes in running mechanics," Journal of Biomechanics, Article vol. 49, no. 14, pp. 3362–3367, 2016, pmid:27616268
- 21. Mason R. et al., "Wearables for Running Gait Analysis: A Systematic Review," (in eng), Sports Med, Oct 15 2022, pmid:36242762
- 22. Strohrmann C., Harms H., Kappeler-Setz C., and Troster G., "Monitoring Kinematic Changes With Fatigue in Running Using Body-Worn Sensors," Ieee Transactions on Information Technology in Biomedicine, vol. 16, no. 5, pp. 983–990, Sep 2012, pmid:22677321
- 23. Goldsack J. C. et al., "Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs)," npj Digital Medicine, vol. 3, no. 1, p. 55, 2020/04/14 2020, pmid:32337371
- 24. Chan A. W. et al., "SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials," (in eng), Bmj, vol. 346, p. e7586, Jan 8 2013, pmid:23303884
- 25. Kim B. H., Hong S. H., Oh I. W., Lee Y. W., Kee I. H., and Lee S. Y., "Measurement of Ankle Joint Movements Using IMUs during Running," (in eng), Sensors (Basel), vol. 21, no. 12, Jun 21 2021, pmid:34205721
- 26. Sabharwal S. and Kumar A., "Methods for assessing leg length discrepancy," (in eng), Clin Orthop Relat Res, vol. 466, no. 12, pp. 2910–22, Dec 2008, pmid:18836788
- 27. Peake J. M., Kerr G., and Sullivan J. P., "A Critical Review of Consumer Wearables, Mobile Applications, and Equipment for Providing Biofeedback, Monitoring Stress, and Sleep in Physically Active Populations," (in English), Frontiers in Physiology, Review vol. 9, no. 743, 2018-June-28 2018, pmid:30002629
- 28. Powell D., Nouredanesh M., Stuart S., and Godfrey A., "Investigating the AX6 inertial-based wearable for instrumented physical capability assessment of young adults in a low-resource setting," Smart Health, vol. 22, p. 100220, 2021/11/01/ 2021,
- 29. Garner J. C., Parrish L. R., Shaw K. R., Wilson S. J., and Donahue P. T., "Using Motion Sensor Technology to Manage Risk of Injury in a Strength and Conditioning Program for Female Collegiate Athletes," 2020, Early Intervention, ACL Injuries, Knee Injuries, Female Athletes, Wearable Electronic Devices vol. 8, no. 1, p. 6, 2020-01-31 2020,
- 30.
E. Charry, W. Hu, M. Umer, A. Ronchi, and S. Taylor, "Study on estimation of peak Ground Reaction Forces using tibial accelerations in running," 2013 IEEE Eighth International Conference on Intelligent Sensors, Sensor Networks and Information Processing, pp. 288–293, 2013.
- 31. Burke A., Dillon S., O’Connor S., Whyte E. F., Gore S., and Moran K. A., "Comparison of impact accelerations between injury-resistant and recently injured recreational runners," PLOS ONE, vol. 17, no. 9, p. e0273716, 2022, pmid:36084137
- 32.
L. Malisoux, R. Ramesh J Fau—Mann, R. Mann R Fau—Seil, A. Seil R Fau—Urhausen, D. Urhausen A Fau—Theisen, and D. Theisen, "Can parallel use of different running shoes decrease running-related injury risk?," (in eng), no. 1600–0838 (Electronic).
- 33.
T. P. Yamato, A. D. Saragiotto Bt Fau—Lopes, and A. D. Lopes, "A consensus definition of running-related injury in recreational runners: a modified Delphi approach," (in eng), no. 1938–1344 (Electronic).
- 34. Roca-Dols A. et al., "Effect of the cushioning running shoes in ground contact time of phases of gait," Journal of the Mechanical Behavior of Biomedical Materials, vol. 88, pp. 196–200, 2018/12/01/ 2018, pmid:30179793
- 35.
H. J. Grubb, A. "WMA Age-grading calculator 2015." http://www.howardgrubb.co.uk/athletics/wmalookup15.html. (accessed 23/10/2022, 2022).
- 36. Bangor A., Kortum P. T., and Miller J. T., "An Empirical Evaluation of the System Usability Scale," International Journal of Human–Computer Interaction, vol. 24, no. 6, pp. 574–594, 2008/07/29 2008,
- 37. Brooke J., "SUS: A quick and dirty usability scale," Usability Eval. Ind., vol. 189, 11/30 1995.
- 38.
Lewis J. R. and Sauro J., "The Factor Structure of the System Usability Scale," in Human Centered Design, Berlin, Heidelberg, Kurosu M., Ed., 2009// 2009: Springer Berlin Heidelberg, pp. 94–103.
- 39. Liang J. et al., "Usability Study of Mainstream Wearable Fitness Devices: Feature Analysis and System Usability Scale Evaluation," (in eng), JMIR Mhealth Uhealth, vol. 6, no. 11, p. e11066, Nov 8 2018, pmid:30409767
- 40.
Bevan N., Carter J., and Harker S., "ISO 9241–11 Revised: What Have We Learnt About Usability Since 1998?," in Human-Computer Interaction: Design and Evaluation, Cham, Kurosu M., Ed., 2015// 2015: Springer International Publishing, pp. 143–151.
- 41. Jacob A., Zakaria W. N. W., Tomari M. R. B. M., Sek T. K., and Suberi A. A. M., "Wearable flex sensor system for multiple badminton player grip identification," 2017/09/1 2017, vol. 1883, p. 020036, [Online]. Available: https://ui.adsabs.harvard.edu/abs/2017AIPC.1883b0036J
- 42. Napier C., Fridman L., Blazey P., Tran N., Michie T. V., and Schneeberg A., "Differences in Peak Impact Accelerations Among Foot Strike Patterns in Recreational Runners," Frontiers in Sports and Active Living, vol. 4, Mar 2022, pmid:35308593
- 43. Lancaster G. A., Dodd S., and Williamson P. R., "Design and analysis of pilot studies: recommendations for good practice," (in eng), J Eval Clin Pract, vol. 10, no. 2, pp. 307–12, May 2004, pmid:15189396
- 44. Lewis M., Bromley K., Sutton C. J., McCray G., Myers H. L., and Lancaster G. A., "Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back!," Pilot and Feasibility Studies, vol. 7, no. 1, p. 40, 2021/02/03 2021, pmid:33536076
- 45. Expósito-Ruiz M., Pérez-Vicente S., and Rivas-Ruiz F., "Statistical inference: hypothesis testing," (in eng), Allergol Immunopathol (Madr), vol. 38, no. 5, pp. 266–77, Sep-Oct 2010, pmid:20817378
- 46. Faulkner L., "Beyond the five-user assumption: Benefits of increased sample sizes in usability testing," Behavior Research Methods, Instruments, & Computers, vol. 35, no. 3, pp. 379–383, 2003/08/01 2003, pmid:14587545
- 47. Young F., Mason R., Wall C., Morris R., Stuart S., and Godfrey A., "Examination of a foot mounted IMU-based methodology for a running gait assessment," (in English), Frontiers in Sports and Active Living, Brief Research Report vol. 4, 2022-September-06 2022, pmid:36147582
- 48. Bland J. M. and Altman D. G., "Statistical methods for assessing agreement between two methods of clinical measurement," (in eng), Lancet, vol. 1, no. 8476, pp. 307–10, Feb 8 1986. pmid:2868172
- 49. Shrout P. E. and Fleiss J. L., "Intraclass correlations: uses in assessing rater reliability," (in eng), Psychol Bull, vol. 86, no. 2, pp. 420–8, Mar 1979, pmid:18839484
- 50. Cicchetti D. V., "Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology," Psychological Assessment, vol. 6, pp. 284–290, 1994,
- 51. Koo T. K. and Li M. Y., "A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research," (in eng), J Chiropr Med, vol. 15, no. 2, pp. 155–63, Jun 2016, pmid:27330520
- 52. Jaeschke R., Singer J., and Guyatt G. H., "Measurement of health status. Ascertaining the minimal clinically important difference," (in eng), Control Clin Trials, vol. 10, no. 4, pp. 407–15, Dec 1989, pmid:2691207
- 53.
Cohen J., Statistical power analysis for the behavioral sciences. Routledge, 2013.
- 54. Benson L. C., Räisänen A. M., Clermont C. A., and Ferber R., "Is This the Real Life, or Is This Just Laboratory? A Scoping Review of IMU-Based Running Gait Analysis," Sensors, vol. 22, no. 5, p. 1722, 2022. [Online]. Available: https://www.mdpi.com/1424-8220/22/5/1722. pmid:35270869
- 55. Black M. I., Handsaker J. C., Allen S. J., Forrester S. E., and Folland J. P., "Is There an Optimal Speed for Economical Running?," (in eng), Int J Sports Physiol Perform, vol. 13, no. 1, pp. 75–81, Jan 1 2018, pmid:28459289
- 56. Folland J. P., Allen S. J., Black M. I., Handsaker J. C., and Forrester S. E., "Running Technique is an Important Component of Running Economy and Performance," (in eng), Med Sci Sports Exerc, vol. 49, no. 7, pp. 1412–1423, Jul 2017, pmid:28263283
- 57. van Oeveren B. T., de Ruiter C. J., Beek P. J., and van Dieën J. H., "The biomechanics of running and running styles: a synthesis," Sports Biomechanics, pp. 1–39, 2021, pmid:33663325
- 58. Williams K. R. and Cavanagh P. R., "Relationship between distance running mechanics, running economy, and performance," Journal of Applied Physiology, vol. 63, no. 3, pp. 1236–1245, 1987/09/01 1987, pmid:3654469
- 59. Clermont C. A., Benson L. C., Edwards W. B., Hettinga B. A., and Ferber R., "New Considerations for Wearable Technology Data: Changes in Running Biomechanics During a Marathon," Journal of Applied Biomechanics, vol. 35, no. 6, pp. 401–409, Dec 2019, pmid:31629343
- 60. Clermont C. A., Pohl A. J., and Ferber R., "Fatigue-Related Changes in Running Gait Patterns Persist in the Days Following a Marathon Race," Journal of Sport Rehabilitation, vol. 29, no. 7, pp. 934–941, Sep 2020, pmid:31825892
- 61. Maas E., De Bie J., Vanfleteren R., Hoogkamer W., and Vanwanseele B., "Novice runners show greater changes in kinematics with fatigue compared with competitive runners," (in eng), Sports Biomech, vol. 17, no. 3, pp. 350–360, Sep 2018, pmid:28730917
- 62. Andreoni G., Standoli C. E., and Perego P., "Defining Requirements and Related Methods for Designing Sensorized Garments," Sensors, vol. 16, no. 6, 2016, pmid:27240361
- 63. McCann J., Hurford R., and Martin A., "A design process for the development of innovative smart clothing that addresses end-user needs from technical, functional, aesthetic and cultural view points," Ninth IEEE International Symposium on Wearable Computers (ISWC’05), pp. 70–77, 2005.
- 64.
Scataglini S., Andreoni G., and Gallant J., "Smart Clothing Design Issues in Military Applications," in Advances in Human Factors in Wearable Technologies and Game Design, Cham, Ahram T. Z., Ed., 2019// 2019: Springer International Publishing, pp. 158–168.