Statistical significance does not produce any information about magnitude of effect, practical significance, nor clinical significance (1). Null hypothesis statistical testing only yields information about the statistical likelihood of certain results based on certain assumptions about the population studied (2).

Practical clinical significance provides an answer to the question, how effective is the intervention or treatment, or how much change does the treatment cause? Usually expressed as effect size, number needed to treat (NNT), and preventive fraction (3).

Clinical significance is a parameter which measures whether a treatment was effective enough to change a patient’s diagnostic label, in other words, is a treatment effective enough to cause the patient to be normal and no longer qualify for the diagnosis given before the start of treatment. Broadly there are two components. Firstly the patients status after treatment and how much change has occurred (4).

While small differences may be statistically significant they are rarely practically relevant and have minimal clinical significance (5,6).

Just as there are several methods of calculating statistical significance and practical significance, there are a number of ways to calculate clinical significance (3,7,8,9).

White et al. (2011) have not even attempted to perform any of these simple and highly illuminating calculations, nor indeed have they attempted to calculate the numbers needed to treat to produce one person with age sex normal functioning, or perhaps even more telling, the number of patients needed before the treatment produces someone with severe or moderate side effects (NNT or NNH).

The data presented in White et al. showed an 8% improvement in self reported fatigue compared to control and an 8% improvement in self reported functionality compared to control in the CBT and GET groups.

When put to the test however the CBT group showed no objective increase in functionality compared to control and the GET.

The GET group were able to walk an extra few meters in 6 minutes compared to control. This left the patients so disabled that they would qualify for entry into the study!

The White et al. study is a classic example of statistically significant results which have no practical or clinical significance whatsoever.

The results also have no benefit financially. McCrone et al. showed that the number of patients in receipt of benefits due to illness or disability in all treatment arms increased after the 12-month post-randomisation period (table 4).

The cost to society (Table 3) was also greatly underestimated with lost employment calculated at around £14,000 in each arm for the period 2009/10. The Office of National Statistics however calculated that “for the tax year ending 5 April 2010 the median gross annual earnings for full-time employees were £25,900” (10).



REFERENCES

1. Haase, R.F., Ellis, M.V., Ladany, N. (1989). Multiple Criteria for Evaluating the Magnitude of Experimental Effects. Journal of Counseling Psychology, 36(4), 511-516.

2. "Clinical" Significance: "Clinical" Significance and "Practical" Significance are NOT the Same Things. Online Submission, Paper presented at the Annual Meeting of the Southwest Educational Research Association (New Orleans, LA, Feb 7, 2008).

3. Peterson, L. (2008). "Clinical" Significance: "Clinical" Significance and "Practical" Significance are NOT the Same Things. Online Submission, Paper presented at the Annual Meeting of the Southwest Educational Research Association (New Orleans, LA, Feb 7, 2008).

4. Jacobson, N., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12-19.

5. Sattler JM (2008). Assessment of children: Cognitive foundations (5/e). San Diego: Sattler Publications. ISBN 978-0-9702671-6-0

6. Kaufman, Alan S.; Lichtenberger, Elizabeth (2006). Assessing Adolescent and Adult Intelligence (3rd ed.). Hoboken (NJ): Wiley. ISBN 978-0-471-73553-3. Lay summary (22 August 2010).

7. Speer, D. C., & Greenbaum, P. E. (1995). Five methods for computing significant individual client change and improvement rates: Support for an individual growth curve approach. Journal of Consulting and Clinical Psychology, 63, 1044-1048.

8. Peterson, L. (2008). "Clinical" Significance: "Clinical" Significance and "Practical" Significance are NOT the Same Things. Online Submission, Paper presented at the Annual Meeting of the Southwest Educational Research.

9. Hageman, W. J., & Arrindell, W. A. (1999). Establishing clinically significant change: increment of precision and the distinction between individual and group level of analysis. Behaviour Research and Therapy, 37, 1169-1193.

10. ONS. (2010) Annual Survey of Hours and Earnings, 2010 Provisional Results http://www.ons.gov.uk/ons...