Home > Education, Psychology, Science > Game Over

Game Over

Let’s not get off on the wrong foot, here: technology is great. I for one simply love flushing toilets, 24-hour rolling news, and spray-on condoms. But despite the near-unimaginable number of problems solved by stellar inventions like these, technology isn’t always the answer. Bear that in mind as we investigate the shiny, sparkly world of ‘brain training’ computer games.

It’s every parent’s dream – those video games your kids spend all their time playing could be harnessed to improve their brains and make them smarter! Just 20 minutes of ‘brain training’ a day could help their mathematical skills, with a boost in self-confidence to boot! At least those are the findings of a recent study in the British Journal of Educational Technology by David Miller and Derek Robertson.

This is Dr. Kawashima. He is real, and he's done some real research! Just... not into brain training.

Their experiment involved 3 classes of primary school kids, aged 10 or 11, each from a different school near Dundee. They were all tested on a ‘Number Challenge’ test, assessing their computational skills, as well as surveyed for self-esteem levels. Then, the first group all got a Nintendo DS Lite along with the well-known Dr. Kawashima’s Brain Training game, which they used for 20 minutes at the start of every school day for 10 weeks. The unfortunate second group didn’t get a Nintendo, but did get a programme of the notorious Brain Gym, the set of bizarre, pulled-out-of-the-inventor’s-arse exercises purported to do wonderful things like ‘increase circulation to the frontal lobe for greater comprehension and rational thinking’ and ‘facilitate whole-brain learning’ – whatever that is. Every morning for the 10 week period, these poor kids contorted themselves into daft positions for 20 minutes, in the name of science. The third group, the controls, didn’t get a cool Nintendo, but weren’t punished with silly old Brain Gym either – silver linings and all that. They continued with their normal school activities. After 10 weeks, all the groups were tested on the Number Challenge and the self-esteem test again.

It turned out that the gaming group had a significant increase in their accuracy on the Number Challenge – an average of 10% better! Wow! But wait – so did the control group, which was an average of 5% better. In contrast, the Brain Gym group showed no significant change in accuracy (wait, doesn’t this mean that Brain Gym is detrimental to kid’s maths learning? Ha!) From this, the authors concluded that, since the score increase was bigger in the gaming group than in the controls, Brain Training really did make a difference to the kid’s computation scores.

Just a second. Almost unbelievably, and as was pointed about by Sergio Della Sala in a letter to the journal, the authors didn’t perform a comparison across the groups. The whole idea of doing scientific experiments, then subjecting them to statistical analysis, is to make sure your results haven’t just appeared by random chance (i.e., they are statistically significant). The question is this: what if it was mere chance that made the gamers’ scores larger than the controls’? The authors simply can’t answer this on the basis of their study, because they forgot to test for it.

What about self-esteem? On one of the measures of the self-esteem questionnaire, the gaming group showed an increase.But is it really a surprise that a group of kids who were given a cool games console – which the authors admit was ‘at that time widely advertised in the media’ – felt better about themselves?

It’s been known since the 1950s that people’s performance in an experiment can increase simply because they’re being studied – a variant of the placebo effect known as the Hawthorne effect. How do we know that this didn’t occur in this experiment? To really equalise the conditions, the control group should have been allowed 20 minutes per day on a different Nintendo DS game. Then, while removing variation caused by one group having a desirable item and one group going without, we’d see if it’s really Dr. Kawashima’s Brain Training that’s making a difference, here.

How old is your brain? How crap is your research?

So, not really the stuff of dreams after all. But now it gets even worse for brain training games. Instead of a crap study apparently showing that they work, a really good study has been published showing that they are useless. In conjunction with the BBC’s pop-science programme, Bang Goes the Theory (which, yes, I did pan in my first ever blog post), Adrian Owen and colleagues were able to recruit a massive 11,430 participants to do some online brain training.

All the participants were given a battery of cognitive tests, looking at skills like verbal short-term memory, reasoning, and spatial working memory. They were then split into three groups, the first of which did some regular planning and problem-solving tasks, the second of which did some regular ‘brain training’-type tasks (similar to those in Dr. Kawashima’s…), and the third of which regularly answered a bunch of general knowledge questions (this latter group were the controls). After 6 weeks, they were all given the cognitive tests again.

Guess what? The brain training group got better at the tests… and so did the problem-solving group… and so did the controls. Everyone improved to just about the same level, no matter what sort of training they had done. They did, however get better and better scores at the training tasks (and the controls got better and better at general knowledge questions). So here we have a study with a huge sample size showing that brain training games make you better at brain training games, and nothing more. Not really much of a surprise there, I hope you’ll agree.

So it seems that if you’re an adult, you’ll get no benefits from an expensive ‘brain training’ game that you won’t get from playing the quiz machine down the pub every so often. If you’re a parent, there’s no convincing evidence that the games will help your kids learn. After the abject failure of fish oil pills, the arrant nonsense of Brain Gym, the tragic story of ‘learning styles’ and now the dodgy claims of brain training, the quest for a magic bullet for learning and education continues. When will they learn…?

_______________
ResearchBlogging.org

Miller, D., & Robertson, D. (2010). Using a games console in the primary classroom: Effects of ‘Brain Training’ programme on computation and self-esteem British Journal of Educational Technology, 41 (2), 242-255 DOI: 10.1111/j.1467-8535.2008.00918.x

Owen, A., Hampshire, A., Grahn, J., Stenton, R., Dajani, S., Burns, A., Howard, R., & Ballard, C. (2010). Putting brain training to the test Nature DOI: 10.1038/nature09042

  1. No comments yet.
  1. May 14, 2010 at 7:30 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: