more_vert
IN 2009, health authorities around the world were stockpiling neuraminidase inhibitors such as oseltamivir (Tamiflu) in anticipation of a feared influenza pandemic.
Billions of dollars were spent based on the belief the drugs would help prevent and treat the disease, providing a windfall for the pharmaceutical industry, particularly Tamiflu manufacturer Roche.
I’ve written about the Tamiflu saga before.
The original stockpiling decision was based on a 2000 Cochrane Review of eight trials that concluded neuraminidase inhibitors were effective for prevention and treatment of influenza.
But that conclusion has since been undermined by a number of researchers, including Bond University’s Professor Chris Del Mar, in a turnabout that provokes questions about some of the foundations of evidence-based medicine.
The most recent Cochrane Review on the subject —of which Professor Del Mar is one of the authors — concludes the only benefit of oseltamivir for which there is clear evidence is a slight reduction in duration of symptoms (by around 21 hours).
Not really billions of dollars worth of effect there.
So why the different conclusions?
Essentially, it all comes down to the data researchers look at when conducting these kinds of reviews.
For decades, the systematic review of published randomised controlled trials has been considered the gold standard in medical research, and this was what the original Cochrane reviewers did.
By combining data from all published trials on a particular subject, researchers are able to see effects in much larger numbers of people than would typically be included in a single trial, in theory making their conclusions more powerful.
In theory. The problem in reality is a small thing called publication bias.
Some trials are simply more likely to be published than others, potentially skewing the results of this kind of meta-analysis.
It’s not just that industry sponsors are obviously going to be keener to publish trials that show benefits rather than harms. Researchers tend to be more enthusiastic about publishing positive results and journals are more likely to accept those papers too.
So any systematic review of published evidence runs the risk of including a disproportionate number of trials that are positive about the particular drug being studied, while omitting many of those that found no benefit or were inconclusive.
Kind of makes you question the whole enterprise, really.
And that is precisely what Professor Del Mar and colleagues did in revising the Cochrane Review.
They abandoned the published trial reports and went instead to the unpublished clinical study reports (CSRs) the industry relies on when seeking approval for new drugs from regulators — when they could get them, that is (the industry is not always willing to share).
CSRs are huge, lumbering beasts of things — one study found they had a median length of 1000 pages — and they contain far more data than can be found in the public record, including results of unpublished trials and unreported findings from published ones.
It’s interesting to wonder how many other pieces of accepted medical wisdom might be overturned if systematic reviews were routinely based on the data in these documents rather than on published trials.
A group of German researchers last week called for CSRs to be made publicly available, based on their comparison of 101 of them with published trial results for the same drugs.
They found a substantial amount of information on patient-relevant outcomes was missing from the public record, potentially undermining the basis for clinical decision making and for evaluation of comparative effectiveness of rival drugs.
In fact, the CSRs contained more than twice as much information on patient-relevant outcomes — benefits and harms — as could be found in the public domain (86% of outcomes were fully reported in the CSRs, compared with 39% in public sources).
The pharmaceutical industry appears unwilling to share this information without a fight but, if we’re really committed to evidence-based medicine, maybe it should be a condition of licensing that they do.
Jane McCredie is a Sydney-based science and medicine writer.
Fully agree.
2 comments:
the “all trials” campaign (alltrials.net) desrves a mention when discussing this topic
and the role of the major and influential journals? – (including Lancet, NEJM etc); it always troubles me how some of these industry sponsored (and written) trials get published by them. One explanation I have heard is that the writers are required to sign over copyright to the publisher. Then when marketing the product, the pharma company is required to only distribute reprints that are licensed by the publisher at a considerable charge to the purchaser (pharma). A situation of scratching each others backs?
Jane,
A number of independent research groups around the world have access to some pharmaceutical company CSRs, which were mostly obtained through litigation and FOI requests. Many of these CSRs are from trials which have remained unpublished for years. Some are from misreported published trials. Peter Doshi, Tom Jefferson, (co-authors of the Cochrane Review you mention), Kay Dickersin, David Healy and Swaroop Vedula developed a proposal called RIAT (Restoring Invisible and Abandoned Trials – a call for people to publish the findings), which was detailed in the BMJ (BMJ Open) on June 13 this year:
http://www.bmj.com/content/346/bmj.f2865
The RIAT group itself has about 178,000 pages of CSRs and other internal pharmaceutical trial reports. The group advised its intention to work through its CSRs and publish the trial results, unless the relevant pharmaceutical company committed to publishing the results. Very few pharmaceutical companies responded to the group, as requested (Rapid Response to the BMJ within 30 days of publication of the RIAT article).
The RIAT team requested other researchers with access to CSRs to join the effort. A number responded to the article with their intention to do so.
I think that at least 4 journals have agreed to publish RIAT papers, including the BMJ.
Excellent piece!
It would be great if medical/surgical procedures could be reviewed in a similar fashion. Systematic reviews are done based on the articles that surgeons which is not considered to be sound ‘evidence’, – or evidence since I last looked – and often the successes are not, and can not be substantiated by data, or shielded from ‘observational bias’ of the author. It is self-reporting which can border on self-promotion, unfazed by contradictory evidence. This way, theories repeated often enough become ‘standard operating procedures’ in time without much scrutiny from peers.
Good summary Jane. As someone who has been involved in a number of Cochrane reviews perhaps a couple of comments are worth making.
The various Cochrane systemic reviews performed are only as good as the data that can be entered into them & it’s long been recognised by the Cochrane Collaboration that publication bias needs to be catered for. There are a number of mechanisms used by the various CC groups to attempt to eradicate this bias & these are usually made explicit in the methodology section of the review – the boring bits that no-one reads anyway. Prof Del Mar et al have used those methodologies effectively, for which they should be congratulated. They must have been fairly persuasive!
The problem for most Cochrane reviewers is how to easily access the CSRs & other documents that contain the additional information. They aren’t subject to FOI legislation, are often held ‘in commercial confidence’ & may not contain easily extractable data. If you think CSRs are hard to get hold of, try extracting individual patient datasets for analysis!