Introduction

Assumptions of bilateral symmetry in the lower limbs during healthy gait are implicit to much of gait research, and many studies collect and report data from only one side of the body, or report the average both sides [1]. Nevertheless, Sadeghi et al. [1] found evidence for the presence of both symmetric and asymmetric characteristics of kinematic and kinetic aspects of gait among healthy people. Furthermore, several populations, including those diagnosed with Parkinson’s disease (PD) or who have experienced stroke, have unilateral neural pathologies which cause distinct gait asymmetries compared to healthy controls [2–6]. These gait asymmetries are of interest in pathological populations for tracking disease progression or rehabilitation progress [6–8].

Measurements of gait asymmetry can be grouped into two classes, discrete and continuous metrics, which either compare two scalar numbers or two continuous time-series. In this paper, we will focus primarily on the more common class of symmetry measures: discrete symmetry metrics (DSM). Sadeghi et al. [1] and Viteckova et al. [9] published reviews of literature analyzing gait symmetry which identified five discrete symmetry metrics commonly used in the literature [10–14].

The first of these five discrete symmetry metrics, a ratio, was introduced by Seliktar and Mizrahi [10] for analyzing asymmetry of ground reaction forces. Robinson et al. [11] also measured symmetry in ground reaction forces with a new metric, commonly referred to as the symmetry index. Vagenas and Hoshizaki [12] introduced a third discrete symmetry metric while analyzing the asymmetry of lower limb kinematics. Plotnik et al. [13] introduced a new metric to assess the relationship between gait asymmetry and freezing of gait in PD. Another metric was proposed by Zifchock et al. [14] to address issues of previous DSMs related to the choice of reference value [11]. Additionally, a research group from Newcastle University uses another metric on a number of spatiotemporal gait variables [15, 16], and two new DSM have been recently proposed by Queen et al. [17] and Alves et al. [18]. Despite their conceptual equivalence, the different defining equations for all of these DSM produce unique numeric results which are not directly comparable [19].

Likewise, these metrics produce different standardized effect sizes and findings of significance for the same underlying effect of asymmetry [17–20]; these results suggest that metrics are not equally powered to detect effects of asymmetry [21]. Achieving adequate power (e.g. power > 0.8) is an important factor in experimental design [21], therefore, understanding the differences in DSM sensitivity to effects of asymmetry may allow the design of better powered experiments. No previous studies comparing common DSM were designed to directly assess metric sensitivity, in terms of power to detect an effect of asymmetry.

Statistical power is a function of sample size, significance criterion, and the effect size [21]. A power analysis known as “sensitivity power analysis” in G*Power [22] can analytically calculate power for many statistical tests given a significance criterion, sample size, and effect size. However, symmetry metric sensitivity cannot be assessed in this way because it remains unclear how a given underlying effect of asymmetry translates to standardized effect sizes for each symmetry metric [18]. Additionally, the sensitivity of various symmetry metrics cannot be precisely assessed using (finite) experimental data as a reference effect of asymmetry because observed effect sizes have inherent error, as point estimates, which would propagate to estimates of sensitivity [21]. Data from observational studies which lack known true effects, as in some previous studies [17, 19, 20], are particularly ill-suited for comparing symmetry metric sensitivity because false positives could overestimate metric sensitivity.

To ensure applicability to a range of populations and study designs, the sensitivity of symmetry metrics should be assessed on a wide range of effects. Precise estimates of power require large amounts of data that could be prohibitive to collect experimentally. A Monte Carlo simulation is a convenient method to generate the large volume of data needed to accurately estimate power while ensuring that a broad range of factors of effect size (e.g. mean difference and group/condition variability) are evaluated.

Therefore, as previous studies have covered the literature with respect to the validity of general assumptions about the degree of asymmetry present in gait [1], and with respect to the breadth of commonly used symmetry metrics and data analyzed for gait symmetry [9], the purpose of this study is to provide a comprehensive assessment of the sensitivity of common discrete symmetry metrics using power simulations and unreported data from a previous study [23] to validate simulation assumptions. This study will aid the design of future studies by informing authors which metric(s) are appropriate to maximize the power to detect asymmetry for their experimental design.

Materials and methods

Results

General metric characteristics

All discrete symmetry metrics evaluated here are shown in Table 1, including their defining equations and important characteristics (the value for perfect symmetry, the limits of the functions for positive inputs, and whether the metric has a directional output—an output which uniquely signifies which input, x or y, is larger/smaller). The order of x and y arguments have been adjusted, when necessary, such that a larger y always produces a positive asymmetry value for every metric. Symmetry metrics are abbreviated as the first three letters of the first author followed by the last two digits of the publication year, and metrics with an absolute value are distinguished by an absolute value function annotation, e.g. “abs(Plo05)”.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Discrete symmetry metrics.

https://doi.org/10.1371/journal.pone.0268581.t001

Power simulation

As sample size increases and group/condition variability decreases, all metrics reach sufficient power at a lower asymmetry magnitude (Fig 1A vs 1B and 1A vs 1C). When R = 1 (a true null effect), average power was approximately equal to the critical alpha, 0.049 ≅ α. Six metrics (Rob87, Vag92, Plo05, Zif08, Que20, and Alv20) display practically equivalent sensitivity for all variabilities and sample sizes (Fig 1). The ratio (Sel86) ceases to asymptotically approach 100% power for variability greater than 0.25, regardless of sample size; at the largest sample size (100) and variability (1.0), sufficient power was not reached for the largest asymmetry magnitude (power = 0.79). In contrast, the abs(Plo05) metric approached 100% power for the entire range of variabilities, but increased in power much slower than the ratio. The slow increase in power of the Sel86 and abs(Plo05) metrics compared to all other metrics was practically significant for all sample sizes, and increased with variability (Figs 1 and 2). The abs(Plo05) and abs(Roc14) metrics both show large decreases in power compared to their respective non-absolute valued versions and increase in power with asymmetry magnitude much slower in comparison (Fig 1). The Roc14 and Alv20b metrics reached sufficient power quicker than other metrics for all variabilities and sample sizes, however meaningful differences between these and other metrics only exist for variability greater than 0.25. The Alv20b metric (with the weighting factor) increases in power faster than the unweighted (Alv20) version, however, this difference is only practically relevant for variabilities greater than 0.5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Estimated power vs. asymmetry magnitude.

Estimated power for (A) a sample size of 15 and a variability of 0.50. (B) a sample size of 35 and a variability of 0.50. (C) a sample size of 15 and a variability of 0.32.

https://doi.org/10.1371/journal.pone.0268581.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Minimum sample size to achieve sufficient power (80%).

Shown at a variability of (A) 0.50 and (B) 0.32. The maximum and minimum sample tested were 100 and 10, respectively. Lower asymmetry magnitude and smaller N (i.e. closer to the bottom left corner) represents higher sensitivity.

https://doi.org/10.1371/journal.pone.0268581.g002

Experimental results

Gait trials from fifteen healthy, young adults (7 female, 23.4 ± 2.8 years (mean ± s.d.); 72.3 ± 13.5 kg; 170.2 ± 8.1 cm) were analyzed [23]. Table 2 reports the average and standard deviation of the swing time, calculated from 140 strides per trial, and the equivalent magnitude asymmetry and variability in the format used in the power simulations. Left and right swing times for both tied and split-belt conditions passed the Anderson-Darling test of normality (p > 0.46). Both tied and split-belt conditions passed Bartlett’s test of equality of variances between the left and right sides (p > 0.44). Results for swing time asymmetry evaluated with all metrics are reported in Table 3. All metrics produced significant differences (p < .001).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Swing time.

https://doi.org/10.1371/journal.pone.0268581.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Swing time asymmetry.

https://doi.org/10.1371/journal.pone.0268581.t003

Discussion

The results of the power simulation follow several basic principles of statistical power: power for a true null effect is equivalent to the critical alpha, power increases with effect size (e.g. increases in asymmetry magnitude and decreases in variability) and sample size [21]. Our results confirm that symmetry metrics assessed in this study exhibit different power for the same underlying effects of asymmetry, particularly for small effects. Two metrics (Roc14 and Alv20b) were highly sensitive and robust to increased variability, two metrics (Sel86 and abs(Plo05)) show poor sensitivity to small effects and high variability. Finally, six metrics (Rob87, Vag92, Plo05, Zif08, Que20, Alv20) exhibited similar sensitivity, at slightly less than the non-absolute value Roc14 and Alv20b metrics. Additionally, the swing time asymmetry results validated assumptions made for the power simulation (independent and normally distributed inputs with equal variances) and were situated within the range of asymmetry magnitudes, variabilities, and sample sizes tested in the power simulation.

The use of the split-belt treadmill to mechanically induce asymmetric changes in swing time prevents the occurrence of false positive test statistics and allows greater confidence in assessing sensitivity based on differences in findings of significance among the metrics. Despite this, several aspects of the swing time results demonstrate the need for power simulations to assess metric sensitivity. First, the large effect sizes demonstrate that every metric was highly powered to detect the effect of asymmetry in swing time, and the unanimous findings of significance for this large effect of asymmetry does not preclude disagreement at smaller effect sizes (i.e. any differences in metric sensitivity are ambiguous in this data). This ambiguity in metric sensitivity due to large effect sizes is similarly apparent when effect size (Cohen’s d) is manually calculated from several gait variables in the results of Patterson et al. [19, Table 1]. Second, all things being equal (sample size, significance criterion), power is a direct function of effect size [21]. However, the inherent uncertainty of observed effect sizes is emphasized by the mismatch between the order of effect sizes in Table 3 and the general results of the power simulation: The abs(Plo05) and Roc14 metrics exhibit the two smallest effect sizes for swing time, despite the power simulation showing large differences in power between these metrics for small asymmetry magnitudes with non-negligible variability. Similarly, the ratio (Sel86) produced the largest effect size, which could falsely support an interpretation of high sensitivity, but a key behavior of the ratio (lack of robustness to high variability compared to other metrics, see Fig 1) is not apparent in the swing time results, due to the low group variability in swing time. The poor sensitivity of the Sel86 metric in the presence of non-negligible variability is due to the definition of symmetry as a ratio. The distribution of a ratio of two normally distributed random variables is a Cauchy distribution—which has an analytically undefined variance. In practice, this can lead to a large variance that obscures mean differences that might otherwise be significant.

In contrast, the power simulation suggests that the Rochester et al. [16] metric without the absolute value, a simple difference, is the most sensitive, with a caveat that it is the only metric which is not normalized by a reference value. The weighted Alves et al. [18] metric is a normalized alternative that has practically equivalent sensitivity. All other symmetry metrics evaluated here produce relative bilateral differences, such that the asymmetry of S(x, y) ≠ S(x + c, y + c); this is also called “scale invariance” by Alves et al. [18] who asserted that symmetry metrics should display this behavior. The lack of scale invariance by the Roc14 metric may be mitigated in some cases via the addition of a covariate to a statistical test. Alternately, all the other metrics assessed here [11, 12, 14, 17, 18] are scale invariant, practically equivalent in sensitivity to effects of asymmetry, and only slightly less sensitive than the metric proposed by Rochester et al. [16].

A strength of this study is that the parameters (asymmetry magnitude and variability) of the power simulation are the major factors in effect sizes, and therefore the power simulation essentially simulated different effect sizes. This allows the results of the power simulation to generalize to more statistical tests than the paired t-test used here. However, real data may not exhibit all the characteristics (asymmetry magnitudes and variability) and assumptions (independent and normally distributed inputs with equal variances, asymmetric data compared to symmetric baseline). Our simulation code and results are available in Zenodo and can be further explored or expanded to test characteristics or assumptions not made here [27].

Conclusion

In this study, we compared commonly used discrete symmetry metrics using power simulations and real data to demonstrate that metrics exhibit different sensitivities to the same underlying effects of asymmetry. Two metrics, published by Rochester et al. [16] (when used without an absolute value) and Alves et al. [18], display excellent sensitivity to a broad range of data characteristics. However, some metrics display very poor sensitivity when data is highly variable, therefore we suggest that future studies consider metric sensitivity to reduce the possibility of underpowered research.

Acknowledgments

The authors would like to thank Tarique Siragy and Mary-Elise MacDonald for proof-reading drafts of this manuscript.