Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

A Generalized Formula for Converting Chi-Square Tests to Effect Sizes for Meta-Analysis

  • Michael S. Rosenberg

    msr@asu.edu

    Affiliation Center for Evolutionary Medicine and Informatics, The Biodesign Institute and School of Life Sciences, Arizona State University, Tempe, Arizona, United States of America

A Generalized Formula for Converting Chi-Square Tests to Effect Sizes for Meta-Analysis

  • Michael S. Rosenberg
PLOS
x

Abstract

The common formula used for converting a chi-square test into a correlation coefficient for use as an effect size in meta-analysis has a hidden assumption which may be violated in specific instances, leading to an overestimation of the effect size. A corrected formula is provided.

Introduction

One of the fundamental concepts in systematic and comparative reviews such as meta-analysis is that of the effect size. An effect size is a statistical parameter that can be used to compare, on the same scale, the results of different studies in which a common effect of interest has been measured [1], [2]. In experimental studies, the effect size is a measurement of the response of the subjects to an experimental treatment relative to a control group. All effect size measures are a means of representing the results of primary research in a common way so that the results from individual studies can be compared and evaluated [1]. While a number of alternate metrics have been suggested for measuring effect size, including standardized mean differences and odds ratios [3], [4], historically, one of the more popular measures of effect has been the correlation coefficient [5], [6], [7], [8], [9], [10]. The correlation coefficient is widely used, easily interpretable, and has the added bonus of being easily determinable from other commonly used statistics such as z-scores, t-tests, F-statistics, and χ2 statistics [3], [11]. These conversions can only be performed for single, focused contrasts (e.g., cases with a single degree of freedom), but otherwise follow simple equations. For example, the equation for converting a χ2 into a correlation [11], [12] is:(1)where the χ2 value comes from a two-group contrasts (thus a single degree of freedom) and n is the total number of samples; the sign of the correlation needs to be determined from independent study of the contrast. (χ2 tests with more than one degree of freedom are unfocused omnibus tests, and require a much more complicated procedure for conversion to an effect size; see [13], [14], [15] for details). Equation (1) has been used to convert χ2 tests into a correlation coefficient for use as an effect size for 45 years [12]; unfortunately, this equation has an underlying, never-stated assumption which is sometimes violated, particularly for genetics studies: it assumes that the expected values from the χ2 test are equal for both groups.

Results and Discussion

Recall that the χ2 is calculated simply as(2)where Oi and Ei are the observed and expected counts, respectively, for group i. Equation (1) assumes that the expected values for the two groups are equal, that is, E1 = E2 = n/2. For the vast majority of χ2 tests this assumption probably holds and equation (1) is correct. However, when this assumption is not met (e.g., if the expected values are for a phenotypic ratio of 3∶1 from a standard Mendelian di-heterozygous dominant-recessive cross), equation (1) is incorrect and will overestimate the effect size (occasionally even producing “correlations” greater than one).

The problem is specifically found in the denominator of the equation. Rather than the sample size, the denominator should actually be the maximum possible χ2 value obtainable for that sample size and expected values:(3)

Written this way, the logic of the equation can be interpreted as r2 being equal to the ratio of observed χ2 to the maximum possible χ2, a not atypical way of expressing the variance explained in a sums-of-squares framework. In the common case where one expects an equal distribution of observations among the two groups (E1 = E2 = n/2), this maximum possible value is equal to n (see Methods), making equation (1) correct.

On the other hand, if the expected values for the two groups are not the same, the maximum possible value is larger than n. Specifically, if the ratio of expectations among the two groups is k∶1, where k represents the larger group (k≥1), the maximum possible χ2 value is equal to nk (see Methods). The general form of the conversion of a χ2 value to r should therefore be(4)

As would be expected, this generalized form simplifies to the commonly used equation when the expected ratio of observations is 1∶1 (i.e., k = 1).

As a simple example, imagine a genetic cross of two heterozygous individuals for a pair of alleles with a simple dominant-recessive relationship. The expectation is a phenotypic ratio of 3∶1, in favor of the dominant phenotype. With 22 total offspring, the expected counts are 16.5 and 5.5, respectively. The actual observation is almost the reverse, with 16 found to have the recessive phenotype and only 6 having the dominant phenotype. The χ2 value for this cross is 26.7 (some additional factor is clearly skewing the expected phenotypic ratio). Using the traditional transformation from equation (1), one would determine an effect size r = 1.10, a value outside the acceptable range of r, impossible to interpret or use in a summary analysis (which generally requires use of Fisher's z-transform). Taking into account the adjusted formula due to the assumption of unequal expectations, we instead find an effect size of r = 0.64, still quite strong, but logically interpretable and meaningful.

It is difficult to determine how often, if ever, equation (1) has been misapplied, because one could not simply go through published meta-analyses looking for χ2 conversions, but would rather have to go back to the original studies used in each meta-analysis to determine how the χ2 test was actually calculated. Fortunately, the assumption of equal expectation among groups likely holds for the majority of χ2 tests. However, those who work in fields or encounter data where this assumption is not true (e.g., heredity experiments), need to be aware of the alternate formulation to use for their specific work, particularly as meta-analysis and other statistics that make use of effect size continue to become more common [16].

Methods

Derivation of maximum possible χ2 value for two groups when the expectation is an equal distribution of counts among the groups

The χ2 value is calculated as(5)where Oi and Ei are the observed and expected values for the ith group. In a χ2 test with only two groups with n total observations, if the expectation is an equal distribution of observations among groups, the expected value of each of the two groups will be n/2. The maximum possible χ2 value will be obtained if one group contains all of the observations and the other group contains zero observations, thus(6)

The maximum possible χ2 value for a two group case where the expected distribution of outcomes is evenly distributed among the two groups is simply the total sample size.

Derivation of maximum possible χ2 value for two groups when the expectation is not an equal distribution of counts among the groups

If the expectation among the two groups is not equal, we have to make the following change. The expected ratio between the two groups will be k∶1, where k is how many times larger one group is expected to be relative to the other (k≥1). In this case the expected values are nk/(k+1) and n/(k+1), respectively (Note that if k = 1, we have an expected ratio of 1∶1, leading both expectations to equal n/2 and the special case described above). Given these new expectations, the maximum possible χ2 value will be when the group with the “nk” expectation (the larger expectation) contains 0 observations and the other group (the smaller expectation) contains all n observations. In this case, the maximum possible χ2 value becomes:(7)

Again, if the expectation is even, k = 1, and the maximum χ2 value is simply n.

Acknowledgments

Thanks to M. Lajeunesse and M. Jennions for discussion and comments on early versions of the manuscript.

Author Contributions

Conceived and designed the experiments: MSR. Performed the experiments: MSR. Analyzed the data: MSR. Contributed reagents/materials/analysis tools: MSR. Wrote the paper: MSR.

References

  1. 1. Cooper HM (1998) Synthesizing Research: A Guide for Literature Reviews. Beverly Hills, CA: Sage. 201 p.HM Cooper1998Synthesizing Research: A Guide for Literature Reviews.Beverly Hills, CASage201
  2. 2. Rosenberg MS, Rothstein H, Gurevitch J (In press) Effect sizes: Choices and calculations. In: Koricheva J, Gurevitch J, Mengersen KL, editors. Handbook of meta-analysis in ecology and evolution. Princeton, NJ: Princeton University Press. MS RosenbergH. RothsteinJ. GurevitchIn pressEffect sizes: Choices and calculations.J. KorichevaJ. GurevitchKL MengersenHandbook of meta-analysis in ecology and evolutionPrinceton, NJPrinceton University Press
  3. 3. Rosenberg MS, Adams DC, Gurevitch J (2000) MetaWin. Statistical Software for Meta-Analysis. 2.0 ed. Sunderland, Massachusetts: Sinauer Associates. MS RosenbergDC AdamsJ. Gurevitch2000MetaWin. Statistical Software for Meta-Analysis. 2.0 ed.Sunderland, MassachusettsSinauer Associates
  4. 4. Hedges LV, Olkin I (1985) Statistical Methods for Meta-Analysis. San Diego, CA: Academic Press, Inc. 369 p.LV HedgesI. Olkin1985Statistical Methods for Meta-Analysis.San Diego, CAAcademic Press, Inc369
  5. 5. Britten HB (1996) Meta-analyses of the association between multilocus heterozygosity and fitness. Evolution 50: 2158–2164.HB Britten1996Meta-analyses of the association between multilocus heterozygosity and fitness.Evolution5021582164
  6. 6. Bender DJ, Contreras TA, Fahrig L (1998) Habitat loss and population decline: A meta-analysis of the patch size effect. Ecology 79: 517–533.DJ BenderTA ContrerasL. Fahrig1998Habitat loss and population decline: A meta-analysis of the patch size effect.Ecology79517533
  7. 7. Møller AP, Thornhill R (1998) Bilateral symmetry and sexual-selection: A meta-analysis. American Naturalist 151: 174–192.AP MøllerR. Thornhill1998Bilateral symmetry and sexual-selection: A meta-analysis.American Naturalist151174192
  8. 8. Reed DH, Frankham R (2001) How closely correlated are molecular and quantitative measures of genetic variation? A meta-analysis. Evolution 55: 1095–1103.DH ReedR. Frankham2001How closely correlated are molecular and quantitative measures of genetic variation? A meta-analysis.Evolution5510951103
  9. 9. Koricheva J (2002) Meta-analysis of sources of variation in fitness costs of plant antiherbivore defenses. Ecology 83: 176–190.J. Koricheva2002Meta-analysis of sources of variation in fitness costs of plant antiherbivore defenses.Ecology83176190
  10. 10. Rosenthal R, Rubin DB (1978) Interpersonal expectancy effects: The first 345 studies. Behavioral and Brain Sciences 3: 377–386.R. RosenthalDB Rubin1978Interpersonal expectancy effects: The first 345 studies.Behavioral and Brain Sciences3377386
  11. 11. Rosenthal R (1994) Parametric measures of effect size. In: Cooper H, Hedges LV, editors. The Handbook of Research Synthesis. New York: Russell Sage Foundation. pp. 231–244.R. Rosenthal1994Parametric measures of effect size.H. CooperLV HedgesThe Handbook of Research SynthesisNew YorkRussell Sage Foundation231244
  12. 12. Cohen J (1965) Some statistical issues in psychological research. In: Wolman BB, editor. Handbook of Clinical Psychology. New York: McGraw-Hill. pp. 95–121.J. Cohen1965Some statistical issues in psychological research.BB WolmanHandbook of Clinical PsychologyNew YorkMcGraw-Hill95121
  13. 13. Rosenthal R, Rosnow RL (1985) Contrast analysis: Focused comparisons in the analysis of variance. New York: Cambridge University Press. R. RosenthalRL Rosnow1985Contrast analysis: Focused comparisons in the analysis of variance.New YorkCambridge University Press
  14. 14. Rosenthal R, Rosnow RL (1991) Essentials of Behavioral Research: Methods and Data Analysis. New York: McGraw-Hill. R. RosenthalRL Rosnow1991Essentials of Behavioral Research: Methods and Data Analysis.New YorkMcGraw-Hill
  15. 15. Rosenthal R, Rosnow RL, Rubin DB (2000) Contrast and effect sizes in behavioral research: A correlational approach. Cambridge: Cambridge University Press. 212 p.R. RosenthalRL RosnowDB Rubin2000Contrast and effect sizes in behavioral research: A correlational approach.CambridgeCambridge University Press212
  16. 16. Koricheva J, Gurevitch J, Mengersen KL, editors. (In press) Handbook of meta-analysis in ecology and evolution. Princeton, NJ: Princeton University Press. J. KorichevaJ. GurevitchKL MengersenIn pressHandbook of meta-analysis in ecology and evolution.Princeton, NJPrinceton University Press