Confidence in research shattered

A REANALYSIS of evidence supporting the anti-influenza drug oseltamivir has undermined confidence in published research for one of the review authors, who has called for an overhaul of the way systematic reviews are conducted.

Professor Chris Del Mar, professor of public health at Bond University, Queensland, was one of seven Cochrane researchers who reanalysed the evidence for oseltamivir (Tamiflu) using primary trial data, much of which was unpublished. (1)

They found several inconsistencies with published reports, such as that oseltamivir did not reduce hospitalisations. The reanalysis also showed a possible underreporting of adverse events, although the drug was found to reduce duration of influenza symptoms by 21 hours.

Governments globally had spent billions of dollars stockpiling oseltamivir after a previously published analysis, funded by Roche, found that the drug reduced complications and hospital admissions. However, eight out of the 10 trials used in the Roche analysis were unpublished and their data sets were not available from either the authors or Roche. (2)

The researchers postponed an analysis of zanamivir (Relenza) evidence because its manufacturer, GlaxoSmithKline offered to provide individual patient data.

The Cochrane researchers found that 60% of patient data from oseltamivir trials had never been published, a fact Professor Del Mar described as “disgraceful”.

In what Professor Del Mar believes is a world first, the researchers obtained original clinical study reports — which can run to tens of thousands of pages — by approaching national regulators, such as the European Medicines Agency and the US Food and Drug Administration (FDA).

Oseltamivir manufacturer Roche also provided data; however, the pharmaceutical company did not provide the full data despite five requests from the researchers.

Professor Del Mar told MJA InSight he believed the current system of systematically reviewing only published literature “isn’t up to the task”.

“We need to revisit that system, probably based on something more like what the regulatory authorities have, where they look at all the information and make their decision by actually querying the manufacturers’ research”, he said.

However, he acknowledged that the considerable time and manpower involved in analysing primary data was a barrier.

Professor Del Mar said that regulators such as Australia’s Therapeutic Goods Administration also needed increased funding and transparency to improve their analysis of any drug requiring significant public expenditure.

He said that drug companies should make any trial data about a drug’s efficacy and safety publicly available, although he acknowledged that early drug development information was commercial-in-confidence.

Professor Del Mar also suggested that raw data from trials could be posted online, possibly hosted by journals as an addendum to published studies. However, he said this raised questions about data ownership and the ethics of reanalysing other researchers’ data.

In an article in the BMJ, three of Professor Del Mar’s coauthors, including Dr Peter Doshi, of Johns Hopkins University School of Medicine, wrote that open access to all relevant trial data was necessary to make ethical health care decisions. (3)

“We think that full, unabridged and properly anonymised clinical study reports would be a good start”, they wrote.

The researchers — from Australia, Italy, US, UK and Japan — were unable to complete a full meta-analysis because of the incompleteness of data; however, they continue to pursue additional data from Roche and the FDA.

“We’re still grinding away getting stuff. A whole lot of hypotheses have come out of this and made us rethink things. We found all sorts of odd stuff”, Professor Del Mar said.

One of the surprising findings was that the placebo used in many of the trials actually contained active ingredients.

The research has also raised questions about the mechanism of action of oseltamivir.

“One of our team has suggested that oseltamivir’s effect might not to be due to an antiviral action but to do with a general brain suppressant … if it is working that way, then it’s no better than paracetamol but a lot more expensive”, Professor Del Mar said.

- Sophie McNamara

1. Cochrane Database of Systematic Reviews 2012; Issue 1
2. Arch Intern Med 2003; 163: 1667-1672
3. BMJ 2012; 17 January (online)

Posted 23 January 2012