Jane McCredie: A tasty con

YOU may have seen claims earlier this year that eating a daily dose of chocolate could help you lose weight.

The claims were based on research from Germany’s Institute of Diet and Health, which found chocolate with a high cocoa content was a “weight loss turbo”, significantly increasing the success of weight loss diets.

Wonderful news for chocolate-loving dieters everywhere … except that the whole thing was a sting operation designed to trap journalists into reporting bad science.

Science journalist John Bohannon recently revealed he was behind the prank in an essay titled “I fooled millions into thinking chocolate helps weight loss. Here’s how”.

As lead author of the study, Dr Bohannon (he has a PhD in the molecular biology of bacteria) secured its publication in a journal called International Archives of Medicine and helped promote it to media outlets around the world.

The journal, which apparently accepts scientific papers on payment of a fee by the authors and without any recognisable process of peer review, has since removed the paper from its website, but you can read it here.

The study was real (though the impressive sounding institute was not). It was based on participants randomised to a low-carb diet, a diet with chocolate or a control group.

The findings were real too: the chocolate group did lose more weight than the others.

The problem? There were only 15 people enrolled in the trial and it measured a very large number of end points — 18 in total including weight, cholesterol, sodium, blood protein levels, sleep quality, wellbeing and so on.

As Dr Bohannon says in his essay, “that study design is a recipe for false positives”.

“Here’s a dirty little science secret: If you measure a large number of things about a small number of people, you are almost guaranteed to get a ‘statistically significant’ result”, he writes.

The research was actually “a fairly typical study for the field of diet research. Which is to say: It was terrible science”.

After conducting this “terrible science” and getting it published in a journal with an authoritative-sounding name, the next task was to spruik it to the media. There, the team had some success though perhaps not as much as Dr Bohannon suggests in his triumphantly titled essay.

The story made the front page of German newspaper, Bild, a publication better known for its pictures of semi-naked women than for investigative journalism.

It was also featured in several health and lifestyle magazines and made it onto Australian morning television, though more serious media outlets generally ignored it.

Where it really took off was online. The internet likes nothing better than a new diet fad, particularly one that tells us the things we like are actually good for us.

I don’t know if anyone has checked to see if there was a contemporaneous spike in chocolate sales, but it wouldn’t be surprising.

Although Dr Bohannon has made much of journalists’ alleged lack of rigour in reporting the story, the more serious issue to me is the proliferation of so-called scientific journals — some of them operated by major publishers — that accept articles for publication in return for payment of a fee, with little or no assessment of their quality.

Dr Bohannon previously conducted a sting operation on behalf of Science, designed to expose the lack of quality control at many open access journals. More than half the journals targeted accepted a bogus and deliberately flawed paper purporting to show inhibition of cancer growth with a molecule extracted from lichen.

The chocolate study similarly encountered few barriers to publication.

Multiple offers from journals were received within 24 hours of submission. The journal selected — which promises “the highest quality constructive peer-review process” — published it without changing a single word.

Jane McCredie is a Sydney-based science and medicine writer.