Citation: Burch D (2014) What Could Computerized Brain Training Learn from Evidence-Based Medicine? PLoS Med 11(11): e1001758. doi:10.1371/journal.pmed.1001758
Academic Editor: Margaret A. Winker, PLoS Medicine, United States of America
Published: November 18, 2014
Copyright: © 2014 Druin Burch. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: DB is a consulting editor for PLOS Medicine and is paid by the Public Library of Science on a freelance basis.
Competing interests: DB is a consulting editor for PLOS Medicine.
Provenance: Commissioned; not externally peer reviewed
Linked Research Article
This Perspective discusses the following new study published in PLOS Medicine:
Lampit A, Hallock H, Valenzuela M (2014) Computerized Cognitive Training in Cognitively Healthy Older Adults: A Systematic Review and Meta-Analysis of Effect Modifiers. PLoS Med 11(11): e1001756. doi:10.1371/journal.pmed.1001756
Michael Valenzuela and colleagues systematically review and meta-analyze the evidence that computerized cognitive training improves cognitive skills in older adults with normal cognition.
Computerized cognitive training (CCT) “is modestly effective at improving cognitive performance in healthy older adults”, find Michael Valenzuela and colleagues, who systematically review the evidence in this week's PLOS Medicine . This is a conclusion of value to academics in the field and to those with interest in selling training programmes. The value to others depends on how well they understand the conclusion's limits.
CCT has a market approaching a billion dollars a year  and an uncertain evidence base. One company's website offers “…a brain training system built and tested by an international team of more than 100 top neuroscientists”, for helping “your brain to make real improvements” . Does the new review support this conception of computer programmes, which in the past would have been described as games or exercises? Either these companies have discovered a novel means of enhancing cognitive performance, or these programs are no more effective—but considerably more expensive—than Sudoku and crosswords .
Adherents to evidence-based medicine seek reliable proof that interventions make a helping difference, and they attempt to make sure that evidence supports branding. Outside of medicine, consumers are treated as capable of making up their own minds about purchases with less help from regulatory oversight. The participants in the studies reviewed here were not enrolled as patients, and most users of CCT programmes have not been prescribed them by physicians. Being asked to pay money for “standardized computerized tasks with clear cognitive rationale” , a fuller description of CCT given by Valenzuela and colleagues, offers a cooler prospect than that implied by “a faster, sharper brain” .
People driven by fear of cognitive decline might reasonably seek guidance about CCT programmes from their physicians. It is otherwise unclear whether providing advice about CCT to those without related symptoms is the business of doctors, except perhaps in the context of the WHO's famously all-embracing definition of health as a “state of complete physical, mental, and social well-being” . Medicine has, however, developed reliable techniques for determining the effects of complex interventions when those interventions cannot be soundly tested by intuition, observation, and argument. Medicine may have no right to say whether people should use CCT, with what ardour, or in what circumstances, but it is well placed to say how CCT should be tested to see what effects it delivers.
Practising Sudoku may have benefits , but not necessarily in terms of increasing your capacity for foreign languages, driving, drawing, or even crosswords. Doing something repeatedly can make you better at it, which is not the same as saying it makes you better. For that reason, Valenzuela and colleagues' review is of studies assessing how practice at particular tasks transferred to more general ones. They estimate that improvements from group-based CCT “may approximate an average relative improvement of 1 point” on the Mini-Mental State Examination . No outcomes, though, were based on differences in actual activities of living. In line with the evidence, the review was also limited to assessing “change in performance from baseline to immediately post-training”. It could not evaluate whether any of the small changes detected (which may or may not extrapolate to settings outside of specific cognitive tests) persist, even to the next day.
Valenzuela and colleagues show effects that are statistically significant but uncertain in their impact on human capacity and performance. Their review, they note, “provides no indication about the durability of the observed gains, nor their transfer into real-life outcomes such as independence, quality of life, daily functioning, or risk of long-term cognitive morbidity”. Such a careful summary is worthwhile, as much for clarifying what is not yet known as for what is. It suggests that clinical trial methodology remains necessary to examine the proposed benefits of CCT. As bloodletting for infection was accepted as beneficial less than a century ago , evidence rather than belief should serve as the basis for decision making about interventions ,. It is required here.
The review's message regarding the limitations of CCT may fail to reach many who are lured by the promises prominent on training websites. Consumers may not make informed decisions, and those experiencing symptoms of dementia or the fear of such symptoms may mistake CCT products for proven medical interventions. That care needs to be taken over the potential for such misunderstanding is plain. Even clinical situations that involve informed consent may not result in realistic expectations: in a study of patients undergoing coronary angioplasty for stable angina between 2009 and 2011, when the evidence showed it did not prevent death, 90% who consented agreed to the procedure in the belief that it did . Moving closer to CCT, the “Mozart effect” showed how experiments on intelligence  could be rapidly and influentially misinterpreted, leading to the widespread sale and purchase of what were thought to be proven tools for useful cognitive enhancement but which were not .
If cardiologists and patients together cannot interpret clear, high-quality evidence, it is reasonable to worry how a limited evidence base will collide with a public wishing to believe in it and a billion-dollar industry based on encouraging them. The benefits and limitations of CCT need to be clearly demonstrated and communicated if the public are not to invest money and time with unrealistic expectations of what they will gain. For CCT to promise more than it delivers and for it to encourage people to oversimplify such a serious matter as the care of their own minds would be a grave pity. Those doctors who once used animal testicles to refresh human vitality and sexual potency, and to augment intelligence and reverse its age-related decline, made a great deal of money from their work. They also performed it sincerely, showed it to be supported by clear rationale, and demonstrated that it made encouraging changes to short-term outcomes –. The allure of CCT is real but so are the potential harms of mistaking and overestimating what is currently sold in its name.
CCT programmes may or may not be medicine, but to be properly assessed their powers certainly require the tools of evidence-based medicine. Does a billion-dollar gap exist between our knowledge about “standardized computerized tasks with clear cognitive rationale” and the industry selling them? Valenzuela and colleagues' overview of the evidence for CCT in cognitively intact older adults suggests it does. It makes clear what remains to be discovered and suggests promising lines of inquiry. Their paper is of use to those planning thoughtful research in the field. It will not be the fault of the authors if the small effects shown, specific to particular subgroups and short-term outcomes, lead to the marketing departments of profitable companies declaring, with added confidence and effect, that their products are “scientifically proven” and doing so without being legally responsible for consumers or patients mistaking what that means.
Wrote the first draft of the manuscript: DB. Wrote the paper: DB. ICMJE criteria for authorship read and met: DB. Agree with manuscript results and conclusions: DB.
- 1. Lampit A, Hallock H, Valenzuela MJ (2014) Computerized Cognitive Training in Cognitively Healthy Older Adults: A Systematic Review and Meta-Analysis of Effect Modifiers. PLoS Med 11(11): e1001756 doi:10.1371/journal.pmed.1001756.
- 2. (2013 August 10) Brain sells: Commercialising neuroscience. The Economist 408: 56.
- 3. Posit Science (2014) BrainHQ. Available: http://www.brainhq.com/play. Accessed 14 October 2014.
- 4. Ferreira N, Owen A, Mohan A, Corbett A, Ballard C (2014) Associations between cognitively stimulating leisure activities, cognitive function and age-related cognitive decline. Int J Geriatr Psychiatry. E-pub ahead of print doi:10.1002/gps.4155.
- 5. WHO (1946) Preamble to the Constitution of the World Health Organization as adopted by the International Health Conference, New York, 19–22 June, 1946; signed on 22 July 1946 by the representatives of 61 States. Official Records of the World Health Organization, no. 2, p. 100 Entered into force on 7 April 1948..
- 6. Osler W (1920) The Principles and Practice of Medicine. 9th edition. New York & London: Appleton & Co.
- 7. Ellis J, Mulligan I, Rowe J, Sackett DL (1995) Inpatient general medicine is evidence based. A-Team, Nuffield Department of Clinical Medicine. The Lancet 346: 407–410 doi:10.1016/S0140-6736(95)92781-6.
- 8. Chapman G, Talbot N, McCartney D, Tippett V, Burch D (2013) Evidence based medicine—older, but no better educated? The Lancet 382: 1484.
- 9. Kureshi F, Jones PG, Buchanan DM, Abdallah MS, Spertus JA (2014) Variation in patients' perceptions of elective percutaneous coronary intervention in stable coronary artery disease: cross sectional study. BMJ 349: g5309.
- 10. Rauscher FH, Shaw GL, Ky KN (1993) Music and spatial task performance. Nature 365: 611.
- 11. Chabris CF (1999) Prelude or requiem for the ‘Mozart effect’? Nature 400: 826–827 doi:10.1038/23608.
- 12. Voronoff S (1920) Life: A Study of the Means of Restoring Vital Energy and Prolonging Life. Voronoff EB, translator. New York: E.P. Dutton & Co. Available: https://archive.org/details/lifestudyofmeans00vororich. Accessed 14 October 2014.
- 13. Brown-Sequard CE (1889) Note on The Effects Produced on Man by Subcutaneous Injections of a Liquid Obtained from the Testicles of Animals. Lancet 134: 105–107 Available: http://www.usrf.org/news/TRT/Brown-Sequard,%20Lancet,%201889.pdf. Accessed 14 October 2014..
- 14. Miller N, Fulmer BR (2007) Injection, Ligation and Transplantation: The Search for the Glandular Fountain of Youth. J Urol 177: 2000–2005.