more_vert
EVIDENCE-based medicine is the mantra of modern medical practice. But what happens when the evidence presented to doctors and the public is flawed, or even actively misleading?
British doctor and writer, Ben Goldacre, well known for his skewering of various forms of quackery in his book Bad Science, has now turned his attention to the distortions of evidence that put mainstream medical treatment at risk.
Never one for quiet understatement, Goldacre writes in his new book, Bad Pharma, that “the whole edifice of medicine is broken because the evidence we use to make decisions is hopelessly and systematically distorted”.
From antidepressants to statins, from cancer drugs to the influenza treatments stockpiled by governments around the world, commercial interests have misrepresented research to exaggerate benefits and downplay side effects, he writes:
“Drugs are tested by the people who manufacture them, in poorly designed trials, on hopelessly small numbers of weird, unrepresentative patients, and analysed using techniques which are flawed by design … When trials throw up results that companies don’t like, they are perfectly entitled to hide them from doctors and patients, so we only ever see a distorted picture of any drug’s true effects.”
The case of the neuraminidase inhibitors — including influenza drugs oseltamivir (Tamiflu) and zanamivir (Relenza) — is informative.
Billions of dollars worth of these drugs were stockpiled by governments for use in the event of a long-feared outbreak of pandemic avian influenza, based on a belief that they would reduce symptoms and prevent complications and transmission of disease.
The WHO had recommended stockpiling of the drugs after a 2000 Cochrane review found the drugs were effective in preventing and treating influenza, based on eight trials including a total of 1180 adults.
That conclusion has, however, been gradually eroded as subsequent Cochrane reviews have uncovered considerably less flattering unpublished data.
In an updated review published earlier this year, the researchers took the unprecedented step of ignoring the published research entirely to focus on the more complete trial data submitted by manufacturers for regulatory approval.
When they could get it, that is.
Professor Chris Del Mar of Bond University, one of the authors of the 2012 review, has described the sometimes frustrating attempts to get original trial data from Tamiflu manufacturer Roche.
The company’s failure to release complete data despite five requests from the authors of the review, as reported in MJA InSight, led them to wonder “whether the data that weren’t published would give as optimistic a result about the drug’s effect on complications as those that were published”, Professor Del Mar wrote.
The reviewers were able to obtain some, but not all, of the missing data from regulators, leading them to conclude … well, that it was difficult to conclude much at all on the basis of the evidence.
Tamiflu appeared to reduce duration of symptoms slightly (by about 21 hours), but the available data did not allow a credible assessment of its effectiveness in preventing either complications or viral transmission, they found.
Billions of dollars in public health funding have been outlaid on the basis of inadequate, possibly misleading, evidence.
Dr Goldacre would argue that’s just one example of distortion on an “industrial scale”, distortion that sees systematic suppression of data on side effects and allows the approval of “hopeless drugs” without adequate evidence of efficacy.
Whether you accept that diagnosis or not, it is way past time for more transparency in the regulatory process.
Clinicians and patients are entitled to demand public release of the full trial data submitted to regulators, not just the sanitised versions that end up appearing in the published record.
Jane McCredie is a Sydney-based science and medicine writer.
Posted 2 October 2012
A vital point that Dr Goldacre makes in one of his talks is about the habit of drug companies withholding data that don’t suit their ends. This is one of his central points, and undermines Dr Ieraci’s comment above: how can we maintain a database of critical research when it goes unpublished? The example he uses is that if I tossed a coin one hundred times then withheld half the results, I could convince you I had a two headed coin. He points out that if we did this in a single study, it is called fraud, whereas when we do it systematically with many entire trials it is accepted as normal. Both his TED talks make great viewing, and his final message to fix this problem is “tell everyone about it”, so please watch them.
The concept of “evidence” is not as straightforward as many in the medical profession may think. Initially by accident, we came across an irregularity of interpretation in the evidence-based medicine (EBM) literature concerning one particular topic. We investigated this further and found what I would describe as a significant failure of evidence assessments that affected more than one-third of the literature in that area. This misinterpretation of evidence extended into clinical practice recommendations and evidence-based guidelines. What happened was that one component (active ingredient) that presumably influenced trial outcomes was simply overlooked in favour of another component that had been tested concurrently in the same trial arms. In fact, over one-third of clinical trials and guidelines (including very prominent ones) unanimously concluded that trial outcomes were solely due to the other agent (when both had been used together). Whether commercial bias towards this agent has played a role is unclear. The interesting thing is that there were no obvious technical flaws in evidence assessments; e.g. the Cochrane Handbook may have been followed by the letter in the failed assessments. Our interpretation is what has been overlooked is simply the principle of biological plausibility. Our article is here: Maiwald M, Chan ES. The forgotten role of alcohol: a systematic review and meta-analysis of the clinical efficacy and perceived role of chlorhexidine in skin antisepsis. http://dx.plos.org/10.1371/journal.pone.0044277
Pharmaceutical manufacturers have the right to spend their R&D money in any way in which they choose. We, as practitioners and voting members of society, are responsible for the way policy is made and the way we prescribe. We all need to maintain our skills in critical analysis of research, and maintain discussion communities where our prescribing is critically examined by our colleagues. Published research doesn’t turn into practice unless we permit it to. Every medical college should maintain a database of critical research findings in its own area, with unbiased analysis of the relevant papers in that area. There will always be “cutting edge” treatments that people want to access as soon as possible – the difference between “cutting edge” and “scandal” is sometimes just a matter of time, however.
It is time for transparency. There are too many stories of suppressed data, non-published trials, ghost writing and even straight out fraud. Unless those who do this are brought to account then the whole research industry and by some reflection the medical profession is tainted.
This is not about bashing pharma. It is about proper accountability of industry and regulators.
We are not talking about a product that if it doesn’t work we can throw away here. We are talking about maybe millions of people whose health may be affected by possibly dangerous or ineffective drugs. The trials should be replicated by an independent body outside of the researching body. Even then, I quiver at the knees at the possible distortion that may occur from under the counter incentives too good to refuse. We also see it in the police force, we see it in the legal profession, and it gets to a stage that when you have seen it often enough, you begin to not trust anyone anymore. All we can do is to close as many loopholes as possible.
Gary, I don’t quite understand how you can argue that any commercial business like pharma that is trying to sell its wares to governments for public use, paid for with public money, should not have their data carefully scrutinised by independent experts. It seems quite obvious and logical and good practice by any group spending public monies. The majority of research that pharma relies on to identify new products comes from publicly-funded research anyway. No other business designs the studies that assess its new products, decides what endpoints to measure, does the data analysis secretly and ‘in-house’ and then decides what to emphasise and what to delete in the published results. These endpoints may not be the most important parameters anyway and may be altered from the initial study design goals to the final published report. The results are then written up by people who often have vast conflicts of interest with the sponsoring company, who has oversight of the publication before submission. The subsequent editorials are then also often written by ‘experts’ with similar conflicts of interest.
I am a baby-boomer and like my fellows, I want the best and most effective treatments to be available to me when I need them. However, I do not want my children and their fellow taxpayers loaded with having to pay extra taxes, or go without other more effective and vital community services, to pay for ineffective and/or risky new treatments that have only been audited by the company producing and marketing them. It seems a complete “no-brainer.”
Well worth listening to Ben Goldacre at: http://www.ted.com/talks/ben_goldacre_what_doctors_don_t_know_about_the_drugs_t...
Never miss an opportunity to bash evil pharma; it is the regulators who should request & receive all the data not an academic who whinges when the company will not provide him with the data it owns & has spent millions on R&D. In my field recent drug development has changed the lives of patients with Rheumatological disease. The trials were conducted world wide generally in tertiary institutions & carefully scrutinised with now post prescribing surveillance.