Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Validity and reliability of running gait measurement with the ViMove2 system

  • Rachel Mason,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Writing – original draft, Writing – review & editing

    Affiliation Department of Sport, Exercise and Rehabilitation, Northumbria University, Newcastle upon Tyne, United Kingdom

  • Gillian Barry,

    Roles Conceptualization, Methodology, Project administration, Writing – review & editing

    Affiliation Department of Sport, Exercise and Rehabilitation, Northumbria University, Newcastle upon Tyne, United Kingdom

  • Gary Hall,

    Roles Methodology, Project administration, Writing – review & editing

    Affiliation Department of Sport, Exercise and Rehabilitation, Northumbria University, Newcastle upon Tyne, United Kingdom

  • Alan Godfrey,

    Roles Conceptualization, Methodology, Project administration, Writing – review & editing

    Affiliation Department of Computer and Information Sciences, Northumbria University, Newcastle upon Tyne, United Kingdom

  • Samuel Stuart

    Roles Conceptualization, Formal analysis, Methodology, Project administration, Writing – original draft, Writing – review & editing

    samuel2.stuart@northumbria.ac.uk

    Affiliations Department of Sport, Exercise and Rehabilitation, Northumbria University, Newcastle upon Tyne, United Kingdom, Department of Neurology, Oregon Health and Science University, Portland, Oregon, United States of America

Abstract

Running biomechanics have traditionally been analysed in laboratory settings, but this may not reflect natural running gait. Wearable technology has the potential to enable precise monitoring of running gait beyond the laboratory. This study aimed to evaluate the analytical validity and intra-session reliability of temporal running gait outcomes measured by the ViMove2 wearable system in healthy adults. Seventy-four healthy adults (43 males, 31 females, aged 18–55 years) wore the inertial device, ViMove2 on the tibia. Participants ran on a treadmill for one minute at various speeds (8, 10, 12, 14km/hr), completed in a standardised shoe (Saucony Guide Runner). Running gait was measured with the ViMove2 wearable and 3D motion capture (Vicon). Temporal running gait outcomes included ground contact time (GCT) and cadence (steps/min). GCT and cadence from the ViMove2 had face validity with expected changes in outcome with different running speeds, but ViMove2 tended to over-estimate GCT, and under-estimate cadence compared to the reference, especially at slower speeds. GCT demonstrated moderate to good agreement to the reference at speeds >10km/hr, but poor agreement at 8km/hr and within female runners. Cadence had moderate to excellent agreement across speeds compared to the reference. GCT and cadence had excellent reliability across speeds, but at 8km/hr GCT had good agreement between trials. Overall, temporal gait outcomes of GCT and cadence can be measured with the ViMove2, but accuracy and reliability are impacted at slow running speeds and within female runners. Future work is needed to clarify sex or speed-dependent corrections to algorithms / outcomes to aid interpretation and application.

Introduction

Running is one of the most popular sport and recreational activities worldwide, consistently ranking within the top five most favored activities, with participation rates spanning from 7.9 to 13.3% globally across the six regions designated by the World Health Organization (WHO) [1]. Further to this, running is a core component of many sports. The utilization of quantitative running gait analysis, both as a means to reduce injury risks and as a metric for performance evaluation, has received considerable attention in scholarly literature [24]. The analysis of running gait carries substantial practical implications for improving athletic performance and assisting in the diagnosis and monitoring of injuries or medical conditions in clinical settings [57]. Traditionally, running gait analysis has been conducted in laboratory settings employing diverse methodologies, including 2D video analysis [8, 9], 3D motion-capture systems [10], force plates [11], instrumented walkways or mats, and treadmills [1214]. Despite their accuracy in measuring gait parameters, these conventional methods pose challenges such as high equipment costs, the necessity for skilled personnel, and the inconvenience of laboratory visits for participants. Moreover, laboratory-based assessments may not authentically replicate real-world running conditions, thereby limiting their applicability in sports and clinical settings [5, 15].

Wearable technology presents a cost-effective and lightweight solution to address the constraints linked with conventional methods of assessing running gait [16]. The increasing acceptance and adoption of wearable technologies among sports practitioners, athletes, patients, and clinicians are notable, with approximately 90% of runners incorporating some type of technology to track their progress [17]. Wearable technology using accelerometers, gyroscopes, and magnetometers, applied individually or in combination as an inertial measurement unit (IMU), allow quantification of running gait outcomes and have become a viable alternative due to their portability and affordability [4, 18, 19]. However, recent research suggests that while wearable technologies for running gait assessment are gaining popularity, less than 10% of commercially available wearables for gait analysis undergo analytical validation against an accepted ’gold-standard’ (reference tool) [20] and even fewer establish the reliability of gait outcomes derived from wearable technology [19]. Ensuring the analytical validity and reliability of these wearable technologies against reference tools is crucial to guarantee the accuracy and reliability of algorithms providing gait outcomes within specific populations (e.g., healthy, or clinical groups), enabling definitive performance and clinical decisions to be made [21, 22]. Following initial analytical validation within specific cohorts, wearable technology and outcomes can be examined for performance or clinical use in various settings (e.g., lab or outdoor/real-world), contributing to a deeper understanding of running gait in both clinical (e.g., neurological, musculoskeletal, or cardio-pulmonary conditions) [23] and sporting contexts (e.g., performance, fatigue, and injury mechanisms) [18].

The ViMove2 System (DorsaVi, www.dorsavi.com/vimove/) is a relatively low-cost commercially available wearable technology that comprises of tibia-based magnetometer inertial measurement units (MIMUs), which can provide the most reported temporal running gait outcomes (i.e., cadence and ground contact time (GCT)) [19]. However, despite being commercially available, the ViMove2 system has never had analytical validation performed, therefore the specific performance characteristics of the technology (i.e., error rates, accuracy etc.) are unknown. Therefore, this study aimed to; 1) determine the analytical validity of temporal running gait outcomes from the ViMove2 in health adults; and 2) examine the intra-session reliability of temporal running gait outcomes from the ViMove2 system in healthy adults. We hypothesized that ViMove2 would have good analytical validity and reliability of temporal running gait outcome measures.

Methods

The following is a brief overview of the study methodology, for detailed information please see the published protocol; (ClinicalTrials.gov NCT05277181) [24].

Participants

Ethical approval was granted by the Northumbria University Research Ethics Committee (Reference: 33358) and this study conformed to the Declaration of Helsinki. Recruitment took place from 02/01/2022 until 02/09/2022. All participants were supplied with informed consent and gave verbal and written consent before performing treadmill-based testing in Northumbria University’s Gait and Biomechanics Laboratory.

Adult participants with a range of running abilities were recruited from running clubs in the North-East of England. Inclusion criteria required participants to be aged 18 years or older, able to run unassisted for short periods. Prior to testing, all participants completed a questionnaire to provide information pertaining to their demographics, lifetime athletic injury history, medical history, sporting pursuits and running personal bests. Injury was classified as “any muscle, bone, tendon or ligament pain in the lower back/legs/knee/foot/ankle that caused a restriction or stoppage of running (distance, speed, duration or training) for at least 7 days or 3 consecutive scheduled training sessions, or that required the runner to consult a physician or other health professional” [25]. Exclusion criteria included medical history of disability that affects running gait safety or ability to follow instructions/tasks and any known illness or disease that would prevent their participation in strenuous physical activities (e.g., cardio-respiratory conditions or acute COVID-19).

Instrumentation

Wearable technology: ViMove2.

The ViMove2 consists of two MIMUs, each comprising a 3D accelerometer (±16g) and 3D gyroscope (±250 °/s) with a 3D magnetometer all sampling at 100Hz. MIMUs were placed onto the anterior surface of the mid-shank of the right and left tibia, as per manufacturer’s guidelines [26]. Tibia device placement is based on the anthropometric measurement of height and a ruler is used to measure exact placement, which is the midpoint between the knee and ankle along the anterior surface of the tibia [27]. After placement of bilateral tibia devices, the ViMove2 system was calibrated via its ViPerform software. Data were transmitted through wireless channel to a recording and feedback device, from which the data were subsequently offloaded onto a PC for further analysis.

Reference system: 3D motion capture system.

The reference system consisted of a 14-camera 3D motion capture system, distributed around a space of 9.8 × 7.9 × 3.2m3, sampling at 250 Hz (VICON, Oxford, UK). The calibration of the Vicon system was conducted before each participant. Sixteen reflective markers were placed on the participants lower limb before testing, and a static calibration trial was initially collected to form a musculoskeletal model [28]. Using a small amount of double-sided tape, the markers were attached bilaterally to the following landmarks: anterior superior iliac spine, posterior superior iliac spine, mid-lateral thigh, lateral knee joint line, lateral mid-shank, lateral malleoli, calcaneal tuberosity, and base of the second metatarsal.

Participant specific information of weight, height, ankle width, knee width, and leg length (from posterior iliac spine to medial malleolus) were measured and inputted in the lower body model [29]. The Plug-in-Gait (PiG) lower body model was used to analyse movement at the joints and evaluate all parameters [30]. The lower body was modelled as seven segments (one pelvis, two thighs, two shanks, and two feet). A normal gait cycle was defined from the initial heel-to-heel contact with the same limb. Additional information of the PiG calculations can be found on Vicon’s website.

Data processing was performed in Vicon Nexus. All markers were labelled, and marker trajectories were filtered using a fourth order low-pass Butterworth filter via dynamic plug-in gait model with 6 Hz cut-off frequency. Gait identification was achieved through visual inspection of initial contact and toe off for consecutive strides over the trials. The trajectory of the heel and toe markers in the Z plane were examined, so that the minimum of the trajectory of one stride specified the timestamp of initial contact. The traced trajectory of the toe marker was used to specify the toe’s movement, so that the minimum of the trajectory specified the timestamp of a toe-off event. The initial contact and toe-off events of left and right foot steps were combined in order to estimate for each step GCT, and cadence over the duration of the trial [31]. Ground contact time and swing time were defined by the time between initial contact and toe-off events and between toe-off and initial contact events, respectively.

Procedures

Prior to commencing the protocol, participants were provided the opportunity to run on the treadmill (Spirit fitness XT485) at a comfortable speed for a warm-up and to familiarise themselves. Participants were asked to run at five speeds, four of these speeds were standardised (i.e., 8, 10, 12 and 14 km/hr) and one speed was their self-selected speed. If a participant could not reach a certain speed (i.e., 12 or 14km/hr) or did not feel comfortable at that speed, then it was not completed. Self-selected speed was determined by the participant’s 5km personal best. The order of speed was consistent across participants, starting at the slowest speed (i.e., 8km/hr) and progressing to the fastest (i.e., 14 km/hr or 5km pace). Data were collected for 60 seconds at each speed. A period of 60s was chosen as it generally aligns with other similar studies in the field with data capture periods ranging from 20s [32] to 90s [19, 33, 34]. Participants could have breaks between trials or could abort the trial at any time. Participants were provided with a standardised, neutral cushioning running shoe (Saucony Guide Runner) to wear during testing to ensure consistency and remove bias from gait-affecting cushioning within e.g., support cushioning running shoes [35]. The ViMove2 system and 3D motion capture were recorded simultaneously to allow direct comparison of running gait outcomes. To assess intra-session reliability, participants completed the protocol in the same format in a repeated-measures design, within the session (i.e., two trials recorded at each speed).

The outcome measures were temporal running gait characteristics of ground contact time and cadence. Ground contact time was defined as the time elapsed (ms) between initial contact (where the foot first makes contact with the ground) and final/terminal contact (where the foot last leaves the ground). The GCT was averaged over the one-minute trials. Cadence was defined as the number of steps taken during a given time (one-minute trials).

Statistical analyses

Data analysis was conducted in SPSS v27 (SPSS Inc., Chicago, IL, USA). Shapiro-Wilks tests indicated a normal distribution of all data (p<0.05). For all data analysis, a statistical significance was classed as p < 0.05. Intra-class correlations (ICC(2,1)) were defined as poor (<0.50), moderate (0.50–0.75), good (0.75–0.90) or excellent (>0.90) [36]. Analytical validity and intra-session reliability were conducted across the whole group, and broken into different sex sub-groups (males, females).

Analytical validity.

Face validity was conducted by comparing descriptive data on temporal running gait outcomes across different running speeds. Concurrent validity was performed using ICC models that examined absolute agreement of temporal gait outcomes from the ViMove2 system and the reference 3D motion capture system. Mean error were calculated between the ViMove2 system and the reference 3D motion capture data for descriptive purposes and are observed as an accuracy metric in the outcomes. Limits of agreement (LoA) between outcomes were calculated and Bland-Altman plots were used to visualize [37].

Intra-session reliability.

To determine the intra-session reliability of the ViMove2 system, Pearson’s correlation coefficients (r), ICCs(2, 1) and LoA between the two testing time-points were calculated [38].

Results

Participants

A total of seventy-four participants completed the validity aspect of the study (43 males, 31 females; 39.1 ± 12.61 years; 1.71 ± 0.14m; 71.3 ± 15.6kg). Participants exhibited a range of running abilities (5km personal best; 23:21 ± 08:27). Seventy-one participants completed the intra-session reliability aspect (41 males, 30 females; 39.4 ± 12.6 years; 1.72 ± 0.14m; 71.3± 15.8kg; 5km personal best; 23:04 ± 08:29). Some data loss was experienced at 8km/hr due a technical issue with the ViMove2 system and a low-cut off threshold whereby the ViMove2 requires people to be running at 6km/hr for it to be detected as running, yet for many participants running at 8km/hr the ViMove2 system did not detect them as running. Additionally, some dropout during higher speeds was experienced (Table 1). For the validation aspect, a total of 578 trials across all speeds were examined. For intra-session reliability analysis, 270 trials across all speeds were examined.

thumbnail
Table 1. Summary of number of participants across all speeds.

https://doi.org/10.1371/journal.pone.0312952.t001

Ground contact time

Table 2 shows the descriptive gait data statistics from, along with the absolute agreement between the two systems for ICC, LoA (% and 95%) and r values. The agreement between the ViMove2 system and 3D motion capture is displayed in Figs 1 and 2.

thumbnail
Fig 1. Bland-Altman plots illustrating the absolute agreement of GCT between the ViMove2 system and 3D motion capture system during treadmill running at various speeds.

(A) 8km/hr, (B) 10 km/hr, (C) 12 km/hr, (D) 14km/hr.

https://doi.org/10.1371/journal.pone.0312952.g001

thumbnail
Fig 2. Bland-Altman plots illustrating the absolute agreement of cadence between the ViMove2 system and 3D motion capture system during treadmill running at various speeds.

(A) 8km/hr, (B) 10 km/hr, (C) 12 km/hr, (D) 14km/hr.

https://doi.org/10.1371/journal.pone.0312952.g002

thumbnail
Table 2. Mean difference, ICC(2,1), limits of agreement (LOA%), and Pearson correlation between the 3D motion capture reference and the ViMove2 system during running.

https://doi.org/10.1371/journal.pone.0312952.t002

Face validity of GCT was shown via the decrease in time with increased running speed (Table 2 and Fig 1). The concurrent agreement for GCT from ViMove2 system and reference were weakest during running at 8km/hr, demonstrating poor agreement (ICC(2,1) 0.30, LoA% 23.41). For running at 10km/hr and 14km/hr moderate agreement was displayed (ICC(2,1) 0.69 and 0.72, LoA% 16.43 and 15.12, respectively) (Table 2 and Fig 1). Intraclass correlations show good agreement during running at 12km/hr (ICC(2,1) 0.79, LoA% 13.54) (Table 2). GCT was slightly overestimated by the ViMove2 system compared to the reference system, with higher mean differences, particularly at slower speed (Table 2).

Concurrent validity of the ViMove2 system was impacted by sex. Females exhibited poor validity during 10, 12 and 14 km/hr trials (ICC(2,1) 0.39, 0.44 and 0.42, LoA% 20.32, 14.00 and 15.78, respectively), whereas males had good to moderate validity for GCT during 10, 12 and 14 km/hr trials (ICC(2,1) 0.88, 0.72 and 0.62, LoA% 12.52, 13.17 and 14.47, respectively) (S1 Table).

With respect to reliability, intraclass correlations showed good reliability for GCT at 8km/hr (ICC(2,1) 0.82, LoA% 17.6), and excellent reliability (ICC(2,1) >0.900, LoA% 6.13 to 6.56) and excellent reliability across all additional speeds (10, 12, and 14 km/hr) Table 3. Sex had no significant impact on the reliability of the ViMove2 system (S2 Table).

thumbnail
Table 3. Mean difference, ICC(2,1), limits of agreement (LOA%), and Pearson correlation between test and retest for the ViMove2 system during running.

https://doi.org/10.1371/journal.pone.0312952.t003

Cadence

Face validity of cadence measurement with the ViMove2 system was shown with the increased cadence with increased running speed (Table 2 and Fig 2). There was moderate agreement between the ViMove2 and reference system was shown when running at 8 km/hr (ICC(2,1) 0.56, LoA% 31.85). The agreement between systems was good to excellent at speeds of 10 km/hr, 12 km/hr, and 14 km/hr (e.g., ICC(2,1) 0.72, 0.84, and 0.93, respectively, Table 2 and Fig 2). Additionally, at slower speeds the ViMove2 system underestimated cadence compared to the reference system, as mean differences decreased in cadence decreased with increased speed (Table 2 and Fig 2).

Minor variations in concurrent validity were observed for measuring cadence in different sexes. Specifically, running at 14km/hr the females exhibited moderate validity for cadence (ICC(2,1) 0.86, LoA% 8.01), whereas males had excellent validity (ICC(2,1) 0.97, LoA% 2.58) (S1 Table).

Cadence demonstrated excellent reliability across all speeds (ICC(2,1) >0.900, LoA% 5.64 to 12.86) Table 3. Sex did not significantly impact the reliability of the ViMove2 system when measuring cadence (S2 Table).

Discussion

This is the first study to examine the analytical validity and reliability of temporal running gait parameters in healthy adults measured by the commercially available ViMove2 system. Overall, the findings indicated that the ViMove2 system is a valid digital health technology for measurement of GCT and cadence in healthy adults, exhibiting expected alterations in outcomes across different running speeds and being comparable to the established ’gold-standard’ reference technology (i.e., 3D motion capture). The ViMove2 system also demonstrated high levels of intra-session reliability across different speeds and trials. However, there are specific factors that influence the validity and reliability that need to be considered when interpreting results (e.g., speed of running and sex of participant), which are discussed further in this section. This study provides the performance characteristics of the ViMove2 system for temporal running gait outcomes, which can be used to determine whether future results are meaningful (e.g., differences in performance are beyond error (mean bias) in the outcome). Understanding the accuracy and reliability of running gait outcomes from relatively low-cost commercial wearable technology is crucial in providing specific data to determine error rate and will allow calculation of minimal important differences in future work [39], which is vital to implementation in applied settings and to reduce reliance on expert analysis and/or gold-standard, high-cost technologies [40].

Ground contact time

Ground contact time is essential to understand due to the implications on running economy [41]. There was a consistent decrease in GCT with increased running speed across systems in this study, which aligns with previous research and demonstrates face validity of GCT measured by the ViMove2 system in healthy adults [42]. Furthermore, on average, GCT values recorded by the ViMove2 system were slightly higher than the reference system across all running speeds, which indicates a systematic overestimation of the GCT by the wearable system (Table 2 and Fig 1). The difference between the wearable and reference systems may relate to the inherent differences in the measurement technologies, with one system attached to the participant and the other relying on video capture (e.g., devices attached to the participant may be able to detect subtle changes in movement prior to the video capture).

In terms of validity, the ViMove2 system GCT had moderate agreement to the reference standard for speeds above 10km/hr but had poor agreement at 8km/hr. Similarly for reliability, the ViMove2 system had excellent intra-session reliability for speeds >10km/hr but only moderate reliability for running at 8km/hr. Validity and reliability of the ViMove2 system was generally better with faster running speed (Tables 2 and 3, Fig 1), which contrasts similar work in the field [31, 43]. For example, Falbriard et al., (2018) demonstrated that research-grade IMU GCT validity degrades with faster speed and our previous work with the high-end commercial wearable system from DANU Sport Ltd. demonstrated excellent validity and reliability (ICC(2, 1) >0.90) for GCT across treadmill running speeds [31, 43]. The ViMove2 system is a commercial technology that has a proprietary (‘black-box’) algorithm for processing MIMU data, and therefore we are unable to determine the exact reason for inaccurate GCT at slower speeds. However, the lack of accuracy at slower speed likely relates to the underlying algorithm thresholds for initial and final contact being inaccurate, e.g., a lower-bound threshold for gait event detection is possibly impacting the temporal data when running at slower speeds. Indeed, the ViMove2 system instructions require runners to attain a minimum running speed threshold of 6km/hr for accurate detection and data capture.

Several of the participant data collection was lost due to the error of running too slowly (i.e., the system reported not being able to detect the participant), even though they were running at 8km/hr on the treadmill. Additionally, at slow speeds there may be extraneous noise due to inconsistent running patterns or movements that can lead to misidentification of initial or final contact events [42]. Regardless, results highlight the importance of considering running speed when interpreting GCT data from the ViMove2 system, as it should be considered when interpreting results, with a lack of accuracy at speeds of 8km/hr meaning that GCT should not be reported at this speed range (e.g., it may require further work to determine speed dependent correction of results).

GCT validity was different for female and male runners. Females exhibited poorer concurrent validity for GCT compared to males at several running speeds, indicating that the GCT measurement via the ViMove2 system may be less accurate in females (S1 Table). The influence of sex on concurrent validity may relate to the ViMove2 system data analysis algorithm not being able to accurately detect the initial and final foot contacts during running within females who inherently have different running gait patterns and muscle activation than male runners [4446]. With the ViMove2 algorithm being proprietary we cannot fully understand the specific reason for the difference in validity for female runners, so further research is required to investigate and compare different algorithms to ensure fit-for-purpose and valid outcomes for this population.

Discrepancies in GCT may have occurred due to the running protocol, e.g., running on a treadmill running may have influenced timing of movement and landing patterns (i.e., initial and final contact events) particularly at higher speeds [47]. The use of treadmill running in a laboratory environment was necessary as an initial form of analytical validation (e.g., the beginning of any analytical validation requires an extremely controlled protocol and environment to ensure data quality), however, future research should consider validating the device in a more ecologically valid setting.

Cadence

Cadence was shown to increase with faster running speed, which demonstrates face validity of the ViMove2 system for measuring this temporal running gait outcome. Additionally, cadence was measured in healthy adults by the ViMove2 system with moderate to excellent validity (Table 2 and Fig 2). Mean difference (error/bias) in cadence measured by the ViMove2 compared to the 3D motion capture system was low across running speeds (i.e., 2-11steps/min), this is in line with other commercial wearable technologies that reported errors of ≤1% for cadence [48, 49], although higher errors have also been reported [50]. Previous guidelines for step-related wearable metrics suggest that an error of <5% is needed for use in clinical trials, while a 10%–15% error may be acceptable for the general population [51], which indicates that ViMove2 may be appropriate for clinical use. Interestingly, there was a consistent improvement in ICCs and reduction in mean differences between the ViMove2 and reference system for cadence with increased running speed (Table 2), which implied better agreement between the two systems at higher running speeds. The observed improvements in ViMove2 validity for cadence with increased running speed is consistent with previous research, which indicates that gait dynamics at faster speeds tend to be more predictable and reproducible, facilitating improved synchronization and agreement between measurement systems [49, 52]. For example, at lower speeds, variations in step length and duration may contribute to discrepancies in cadence measurements between systems. However, as speed increases, stride length tends to shorten, resulting in more uniform step intervals and facilitating improved synchronization and agreement between the ViMove2 and reference system [5355].

At higher speed (14km/hr), validity of the cadence measurement by ViMove2 was impacted by sex with females only having moderate concurrent validity but males having excellent validity, similar to GCT measurement (S1 Table). Cadence measurement with the ViMove2 system algorithm may be influenced by the specific running patterns of the female and male runners. Specifically, the algorithm may be reliant on the limited lower limb movement in frontal and transverse planes that males show compared to females when running [46, 56], as this may create noise within the signal that is not being controlled for. Further work is needed to understand how different running mechanics may influence algorithm and device validity, as well as how different algorithms may be deployed to overcome limitations.

Cadence measurements demonstrated excellent intra-session reliability across all running speeds, with percentage LoA ranging from 5.64% to 12.86%, suggesting strong agreement between test and retest measurements (Table 3). These findings collectively underscore the reliability of cadence measurements as collected by the ViMove2 system Facross a range of speeds.

Study limitations

This study has several limitations that should be addressed in future research. The first limitation is the validation of running gait during treadmill running in a laboratory, which may influence performance (e.g., running metrics, psychological factors etc.) and may not represent validity in settings where the wearable technology will be used (e.g., outdoor uncontrolled environments). In future studies could perform outdoor validation through comparison to other reference standards (e.g., instrumented insoles, 3D motion capture or 2D video capture etc.), which could then be applied to a range of testing environments [57, 58]. The second limitation was that it is currently not possible for the ViMove2 system and the 3D motion capture system to be electronically synchronized for data collection, therefore offline timestamps were used to ensure that the same steps were included in all analysis. Finally, while the current study included a range of running speeds for validity and reliability testing there are a range of other factors that may be considered in future work. For example, different surfaces, gradients, and direction changes, as well as in clinical populations.

Conclusions

This study investigated the analytical validity and reliability of a commercial low-cost wearable technology, ViMove2, for measurement of temporal running gait outcomes during treadmill running in healthy adults. Overall, the ViMove2 system can measure GCT and cadence with comparable accuracy to a technological reference standard, but accuracy and reliability is affected by running speed and the sex of the participant. The ViMove2 system has poor validity and moderate reliability at 8km/hr, which indicates that speed should be considered when interpreting findings and conducting research. The ViMove2 system also had poor to moderate validity for GCT and cadence respectively for female runners, which may relate to inherent sex-related running mechanic differences and requires further investigation into underlying proprietary algorithms to determine the exact cause. Future work is required to further examine validity of the ViMove2 in more ecologically valid ‘real-world’ settings and to establish minimal important differences for clinical or performance applications.

Supporting information

S1 Table. Mean difference, ICC(2,1), limits of agreement (LOA%), and Pearson correlation between the 3D motion capture reference and the ViMove2 system during running separated by sex.

https://doi.org/10.1371/journal.pone.0312952.s001

(DOCX)

S2 Table. Mean difference, ICC(2,1), limits of agreement (LOA%), and Pearson correlation between test and retest for the ViMove2 system separated by sex.

https://doi.org/10.1371/journal.pone.0312952.s002

(DOCX)

Acknowledgments

The authors would like to thank all those who volunteered as part of this study. The authors very much appreciate the help and work of Fraser Young.

References

  1. 1. Hulteen RM, Smith JJ, Morgan PJ, Barnett LM, Hallal PC, Colyvas K, et al. Global participation in sport and leisure-time physical activities: A systematic review and meta-analysis. Prev Med. 2017;95:14–25. pmid:27939265
  2. 2. Wang L, Hu W, Tan T. Recent developments in human motion analysis. Pattern Recognition. 2003;36(3):585–601.
  3. 3. Chen KY, Janz KF, Zhu W, Brychta RJ. Redefining the roles of sensors in objective physical activity monitoring. Med Sci Sports Exerc. 2012;44(1 Suppl 1):S13–23. pmid:22157770
  4. 4. Tao W, Liu T, Zheng R, Feng H. Gait analysis using wearable sensors. Sensors (Basel). 2012;12(2):2255–83. pmid:22438763
  5. 5. Norris M, Anderson R, Kenny IC. Method analysis of accelerometers and gyroscopes in running gait: A systematic review. Proceedings of the Institution of Mechanical Engineers Part P-Journal of Sports Engineering and Technology. 2014;228(1):3–15.
  6. 6. Mann R, Malisoux L, Urhausen A, Meijer K, Theisen D. Plantar pressure measurements and running-related injury: A systematic review of methods and possible associations. Gait & Posture. 2016;47:1–9. pmid:27264395
  7. 7. Chambon N, Delattre N, Guéguen N, Berton E, Rao G. Shoe drop has opposite influence on running pattern when running overground or on a treadmill. Eur J Appl Physiol. 2015;115(5):911–8. pmid:25501676
  8. 8. Dingenen B, Barton C, Janssen T, Benoit A, Malliaras P. Test-retest reliability of two-dimensional video analysis during running. Phys Ther Sport. 2018;33:40–7. pmid:30005426
  9. 9. Pipkin A, Kotecki K, Hetzel S, Heiderscheit B. Reliability of a Qualitative Video Analysis for Running. J Orthop Sports Phys Ther. 2016;46(7):556–61. pmid:27266886
  10. 10. Pfister A, West AM, Bronner S, Noah JA. Comparative abilities of Microsoft Kinect and Vicon 3D motion capture for gait analysis. Journal of medical engineering & technology. 2014;38(5):274–80. pmid:24878252
  11. 11. Leitch J, Stebbins J, Paolini G, Zavatsky AB. Identifying gait events without a force plate during running: A comparison of methods. Gait & Posture. 2011;33(1):130–2.
  12. 12. Donath L, Faude O, Lichtenstein E, Nüesch C, Mündermann A. Validity and reliability of a portable gait analysis system for measuring spatiotemporal gait characteristics: comparison to an instrumented treadmill. J Neuroeng Rehabil. 2016;13:6. pmid:26790409
  13. 13. Higginson BK. Methods of running gait analysis. Curr Sports Med Rep. 2009;8(3):136–41. pmid:19436169
  14. 14. Parati M, Ambrosini E, B DEM, Gallotta M, Dalla Vecchia LA, Ferriero G, et al. The reliability of gait parameters captured via instrumented walkways: a systematic review and meta-analysis. Eur J Phys Rehabil Med. 2022;58(3):363–77. pmid:34985239
  15. 15. Dugan SA, Bhat KP. Biomechanics and analysis of running gait. Phys Med Rehabil Clin N Am. 2005;16(3):603–21. pmid:16005396
  16. 16. Willy RW. Innovations and pitfalls in the use of wearable devices in the prevention and rehabilitation of running related injuries. Physical Therapy in Sport. 2018;29:26–33. pmid:29172095
  17. 17. DeJong AF, Fish PN, Hertel J. Running behaviors, motivations, and injury risk during the COVID-19 pandemic: A survey of 1147 runners. PLoS One. 2021;16(2):e0246300. pmid:33577584
  18. 18. Stuart S, Powell D, Marshall SJ, Clark CCT, Martini DN, Johnston W, et al. Chapter 15—Sports medicine: bespoke player management. In: Godfrey A, Stuart S, editors. Digital Health: Academic Press; 2021. p. 231–51.
  19. 19. Mason R, Pearson LT, Barry G, Young F, Lennon O, Godfrey A, et al. Wearables for Running Gait Analysis: A Systematic Review. Sports Med. 2022.
  20. 20. Peake JM, Kerr G, Sullivan JP. A Critical Review of Consumer Wearables, Mobile Applications, and Equipment for Providing Biofeedback, Monitoring Stress, and Sleep in Physically Active Populations. Frontiers in Physiology. 2018;9(743). pmid:30002629
  21. 21. Goldsack JC, Coravos A, Bakker JP, Bent B, Dowling AV, Fitzer-Attas C, et al. Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs). npj Digital Medicine. 2020;3(1):55. pmid:32337371
  22. 22. Rochester L, Mazzà C, Mueller A, Caulfield B, McCarthy M, Becker C, et al. A Roadmap to Inform Development, Validation and Approval of Digital Mobility Outcomes: The Mobilise-D Approach. Digit Biomark. 2020;4(Suppl 1):13–27. pmid:33442578
  23. 23. Hulleck AA, Menoth Mohan D, Abdallah N, El Rich M, Khalaf K. Present and future of gait assessment in clinical practice: Towards the application of novel trends and technologies. Front Med Technol. 2022;4:901331. pmid:36590154
  24. 24. Mason R, Godfrey A, Barry G, Stuart S. Wearables for running gait analysis: A study protocol. PLOS ONE. 2023;18(9):e0291289. pmid:37695752
  25. 25. Yamato TP, Saragiotto BT, Lopes AD. A consensus definition of running-related injury in recreational runners: a modified Delphi approach. J Orthop Sports Phys Ther. 2015;45(5):375–80. pmid:25808527
  26. 26. Garner JC, Parrish LR, Shaw KR, Wilson SJ, Donahue PT. Using Motion Sensor Technology to Manage Risk of Injury in a Strength and Conditioning Program for Female Collegiate Athletes. 2020. 2020;8(1):6.
  27. 27. Charry E, Hu W, Umer M, Ronchi A, Taylor S. Study on estimation of peak Ground Reaction Forces using tibial accelerations in running. 2013 IEEE Eighth International Conference on Intelligent Sensors, Sensor Networks and Information Processing. 2013:288–93.
  28. 28. Kim BH, Hong SH, Oh IW, Lee YW, Kee IH, Lee SY. Measurement of Ankle Joint Movements Using IMUs during Running. Sensors (Basel). 2021;21(12). pmid:34205721
  29. 29. Sabharwal S, Kumar A. Methods for assessing leg length discrepancy. Clin Orthop Relat Res. 2008;466(12):2910–22. pmid:18836788
  30. 30. Leboeuf F, Baker R, Barré A, Reay J, Jones R, Sangeux M. The conventional gait model, an open-source implementation that reproduces the past but prepares for the future. (1879–2219 (Electronic)).
  31. 31. Falbriard M, Meyer F, Mariani B, Millet GP, Aminian K. Accurate estimation of running temporal parameters using foot-worn inertial sensors. Frontiers in physiology. 2018:610. pmid:29946263
  32. 32. McGrath D, Greene BR, O’Donovan KJ, Caulfield B. Gyroscope-based assessment of temporal gait parameters during treadmill walking and running. Sports Engineering. 2012;15:207–13.
  33. 33. Tan T Fau—Strout ZA, Strout Za Fau—Shull PB, Shull PB. Accurate Impact Loading Rate Estimation During Running via a Subject-Independent Convolutional Neural Network Model and Optimal IMU Placement. 2020(2168–2208 (Electronic)).
  34. 34. Bailey G, Harle R. Measuring Temporal Parameters of Gait with Foot Mounted IMUs in Steady State Running2015. 24–33 p.
  35. 35. Roca-Dols A, Losa-Iglesias ME, Sánchez-Gómez R, Becerro-de-Bengoa-Vallejo R, López-López D, Rodríguez-Sanz D, et al. Effect of the cushioning running shoes in ground contact time of phases of gait. 2018(1878–0180 (Electronic)).
  36. 36. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016;15(2):155–63. pmid:27330520
  37. 37. Bland Jm Fau—Altman DG, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. (0140–6736 (Print)).
  38. 38. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86(2):420–8. pmid:18839484
  39. 39. Daoud AI, Geissler GJ, Wang F, Saretsky J, Daoud YA, Lieberman DE. Foot strike and injury rates in endurance runners: a retrospective study. Med Sci Sports Exerc. 2012;44(7):1325–34. pmid:22217561
  40. 40. Young F, Coulby G, Watson I, Downs C, Stuart S, Godfrey A. Just Find It: The Mymo Approach to Recommend Running Shoes. IEEE Access. 2020;8:109791–800.
  41. 41. Di Michele R, Merni F. The concurrent effects of strike pattern and ground-contact time on running economy. J Sci Med Sport. 2014;17(4):414–8. pmid:23806876
  42. 42. Young F, Mason R, Wall C, Morris R, Stuart S, Godfrey A. Examination of a foot mounted IMU-based methodology for a running gait assessment. Frontiers in Sports and Active Living. 2022;4. pmid:36147582
  43. 43. Mason R, Barry G, Robinson H, O’Callaghan B, Lennon O, Godfrey A, et al. Validity and reliability of the DANU sports system for walking and running gait assessment. Physiological Measurement. 2023;44(11):115001. pmid:37852268
  44. 44. Ferber R, Davis IM, Williams DS 3rd. Gender differences in lower extremity mechanics during running. Clin Biomech (Bristol, Avon). 2003;18(4):350–7. pmid:12689785
  45. 45. Rueda J, Chorro D, Torres G, Navandar A, Navarro E. RUNNING PATTERN DIFFERENCES IN GENDER AND RUNNING LEVEL2017.
  46. 46. Xie PP, István B, Liang M. Sex-specific differences in biomechanics among runners: A systematic review with meta-analysis. Front Physiol. 2022;13:994076. pmid:36213228
  47. 47. Van Hooren B, Fuller JT, Buckley JD, Miller JR, Sewell K, Rao G, et al. Is Motorized Treadmill Running Biomechanically Comparable to Overground Running? A Systematic Review and Meta-Analysis of Cross-Over Studies. Sports Med. 2020;50(4):785–813. pmid:31802395
  48. 48. Adams D, Pozzi F, Carroll A, Rombach A, Zeni J Jr. Validity and Reliability of a Commercial Fitness Watch for Measuring Running Dynamics. J Orthop Sports Phys Ther. 2016;46(6):471–6. pmid:27117729
  49. 49. Han H, Kim H, Sun W, Malaska M, Miller B. Validation of wearable activity monitors for real-time cadence. Journal of Sports Sciences. 2020;38(4):383–9. pmid:31818198
  50. 50. Pairot de Fontenay B, Roy J-S, Dubois B, Bouyer L, Esculier J-F. Validating Commercial Wearable Sensors for Running Gait Parameters Estimation. IEEE Sensors Journal. 2020;PP:1-.
  51. 51. Johnston W, Judice PB, Molina García P, Mühlen JM, Lykke Skovgaard E, Stang J, et al. Recommendations for determining the validity of consumer wearable and smartphone step count: expert statement and checklist of the INTERLIVE network. Br J Sports Med. 2021;55(14):780–93. pmid:33361276
  52. 52. Fokkema T, Kooiman TJ, Krijnen WP, VDS CP, DEG M. Reliability and Validity of Ten Consumer Activity Trackers Depend on Walking Speed. Med Sci Sports Exerc. 2017;49(4):793–800. pmid:28319983
  53. 53. Jordan K, Challis JH, Newell KM. Speed influences on the scaling behavior of gait cycle fluctuations during treadmill running. Human Movement Science. 2007;26(1):87–102. pmid:17161484
  54. 54. Jordan K, Newell KM. The Structure of Variability in Human Walking and Running is Speed-Dependent. Exercise and Sport Sciences Reviews. 2008;36(4). pmid:18815489
  55. 55. García-Pinillos F, García-Ramos A, Ramírez-Campillo R, Latorre-Román P, Roche-Seruendo LE. How Do Spatiotemporal Parameters and Lower-Body Stiffness Change with Increased Running Velocity? A Comparison Between Novice and Elite Level Runners. J Hum Kinet. 2019;70:25–38. pmid:31915473
  56. 56. Chumanov ES, Wall-Scheffler C, Heiderscheit BC. Gender differences in walking and running on level and inclined surfaces. Clinical Biomechanics. 2008;23(10):1260–8. pmid:18774631
  57. 57. Benson LC, Clermont CA, Bošnjak E, Ferber R. The use of wearable devices for walking and running gait analysis outside of the lab: A systematic review. 2018(1879–2219 (Electronic)).
  58. 58. Meyer F, Falbriard M, Mariani B, Aminian K, Millet GP. Continuous Analysis of Marathon Running Using Inertial Sensors: Hitting Two Walls? Int J Sports Med. 2021;42(13):1182–90. pmid:33975367