This time last year, if you read any of the newspapers, you’d have undoubtedly read something about the Durham Fish Oil Trial, a purported experiment which meant that thousands of children in Durham were being given fish oil supplements every day, in an effort to show that they improved brain performance. As people pointed out, though, this was anything but a scientific trial: there was no control group, and the data was very tightly held.
“This was an area of failing schools, remember, receiving a huge amount of extra effort and input of all forms. The preceding year, with no fish oil, the results – the number of kids getting 5 GCSE grades A* to C – had improved by 5.5%. And now? After the fish oil intervention? Well. This rate of improvement seems to have deteriorated spectacularly. I chased the results myself through Durham press office: this year there was only a 3.5% improvement. And this is against a backdrop of a 2% increase in GCSE scores nationally anyway. 1.5% over, in an area which was rapidly improving before, and which was receiving huge amounts of extra resourcesand input. You live by anecdata, you die by anecdata: you could argue this ‘trial’ had a negative result.”
Of course, if read the current reports about the trial, you wouldn’t actually know any of this. You’d think it was a massive success. But then, bad non-science backed by an effective PR company trumps real, slow but valuable research every time.