Jane McCredie: Dubious evidence

EVIDENCE-based medicine is the mantra of modern medical practice. But what happens when the evidence presented to doctors and the public is flawed, or even actively misleading?

British doctor and writer, Ben Goldacre, well known for his skewering of various forms of quackery in his book Bad Science, has now turned his attention to the distortions of evidence that put mainstream medical treatment at risk.

Never one for quiet understatement, Goldacre writes in his new book, Bad Pharma, that “the whole edifice of medicine is broken because the evidence we use to make decisions is hopelessly and systematically distorted”.

From antidepressants to statins, from cancer drugs to the influenza treatments stockpiled by governments around the world, commercial interests have misrepresented research to exaggerate benefits and downplay side effects, he writes:

“Drugs are tested by the people who manufacture them, in poorly designed trials, on hopelessly small numbers of weird, unrepresentative patients, and analysed using techniques which are flawed by design … When trials throw up results that companies don’t like, they are perfectly entitled to hide them from doctors and patients, so we only ever see a distorted picture of any drug’s true effects.”

The case of the neuraminidase inhibitors — including influenza drugs oseltamivir (Tamiflu) and zanamivir (Relenza) — is informative.

Billions of dollars worth of these drugs were stockpiled by governments for use in the event of a long-feared outbreak of pandemic avian influenza, based on a belief that they would reduce symptoms and prevent complications and transmission of disease.

The WHO had recommended stockpiling of the drugs after a 2000 Cochrane review found the drugs were effective in preventing and treating influenza, based on eight trials including a total of 1180 adults.

That conclusion has, however, been gradually eroded as subsequent Cochrane reviews have uncovered considerably less flattering unpublished data.

In an updated review published earlier this year, the researchers took the unprecedented step of ignoring the published research entirely to focus on the more complete trial data submitted by manufacturers for regulatory approval.

When they could get it, that is.

Professor Chris Del Mar of Bond University, one of the authors of the 2012 review, has described the sometimes frustrating attempts to get original trial data from Tamiflu manufacturer Roche.

The company’s failure to release complete data despite five requests from the authors of the review, as reported in MJA InSight, led them to wonder “whether the data that weren’t published would give as optimistic a result about the drug’s effect on complications as those that were published”, Professor Del Mar wrote.

The reviewers were able to obtain some, but not all, of the missing data from regulators, leading them to conclude … well, that it was difficult to conclude much at all on the basis of the evidence.

Tamiflu appeared to reduce duration of symptoms slightly (by about 21 hours), but the available data did not allow a credible assessment of its effectiveness in preventing either complications or viral transmission, they found.

Billions of dollars in public health funding have been outlaid on the basis of inadequate, possibly misleading, evidence.

Dr Goldacre would argue that’s just one example of distortion on an “industrial scale”, distortion that sees systematic suppression of data on side effects and allows the approval of “hopeless drugs” without adequate evidence of efficacy.

Whether you accept that diagnosis or not, it is way past time for more transparency in the regulatory process.

Clinicians and patients are entitled to demand public release of the full trial data submitted to regulators, not just the sanitised versions that end up appearing in the published record.

Jane McCredie is a Sydney-based science and medicine writer.

Posted 2 October 2012