Reader Comments
Post a new comment on this article
Post Your Discussion Comment
Please follow our guidelines for comments and review our competing interests policy. Comments that do not conform to our guidelines will be promptly removed and the user account disabled. The following must be avoided:
- Remarks that could be interpreted as allegations of misconduct
- Unsupported assertions or statements
- Inflammatory or insulting language
Thank You!
Thank you for taking the time to flag this posting; we review flagged postings on a regular basis.
closeThe research looked compelling, but I worry about the validity of the statistical analysis
Posted by wsmith924 on 10 Dec 2019 at 17:05 GMT
The following paragraph is found in the article: "Comparison of the relative reductions in velocity between the 1st and 100th place (age groups and event distances aggregated) demonstrated that the girls had greater reductions in relative velocity across place than boys (Fig 3A; sex × place, p<0.001). The average 100th place US record holder swam at 90.7 ± 8.8% the velocity of the first place US record holder for the boys and 89.3 ± 9.0% for the girls (age groups and distances pooled). Thus, the sex difference in swimming performance progressively increased with US record place between first place (boys 2.4% faster) to 100th place (boys 4.3% faster) across all ages and distances (p<0.001). These data indicate that there was less depth of performance in girls than boys (Fig 3). Despite the lesser depth of performance in girls, annual participation was higher in girls compared to boys (p<0.001; Fig 3B)."
What am I not understanding?
1) The paragraph states that girls had a "greater reduction in relative velocity," but the numbers cited (90.7 ± 8.8% vs. 89.3 ± 9.0%) seem to imply a completely insignificant difference. Am I misreading this
2) What is "sex x place"? Sex is a categorical variable, while place is a rank-order variable -- how are they combined?
3) They have already demonstrated that there are very different patterns depending on age groups (where prior to puberty there is little difference between sexes (and the girls are perhaps faster), but after puberty there is an increasingly large difference in performance). Why would they pool the age groups? This introduces a known noise factor.
4) "There was less depth of performance in girls than boys." Would this be expected? Assuming homoscedasticity, if there is a difference in means of bell curves, would one expect a greater drop-off in performance in the lower-performance group than the other? My intuition is the opposite -- that because the probabilistic differences in performance between two groups with different means would be greater the further out in the tails and less further in, I would expect less difference in performance the closer to the mean. But thus far I have been unable to formalize my thoughts on this. Perhaps others reading these comments could help me out. Are the authors arguing against homoscedasticity?
5) In articulating the question in (4), one comes to a realization -- this part of the research seems extremely suspect. It is all based on comparison to the performance of not just the outliers, but of the absolutely *furthest-out outlier* (in each event and age). Normal rules of statistics don't apply in such situations... While it is possible to estimate a mean by looking at the characteristics of a statistical tail of a known sample size from a known total population, comparisons to THE outlier are meaningless. (I guess this might answer question 3, above: They were likely trying to make a population of "absolute outliers" by pooling all of the first place speeds in all events with all ages).
6) How are they comparing the "100th place" of boys vs. girls, when there are different total numbers of competitors? The "100th place" will represent different percentiles. Isn't this a problem?
7) While I do not doubt the general conclusion of the paper, validity of statistical testing and reporting of p-values is based on some fundamental assumptions -- the most basic of which is that the underlying data are properly characterized by a normal distribution. By definition, however, "first place winners" are ABnormal. Before I can believe the statistical inferences, I would need to see more data supporting the idea that the requirements for the statistical tests have been satisfied.