more_vert
TEN years ago, a new regime began in medical publishing that was designed to rein in manipulation of published trial results.
From July 2005, the International Committee of Medical Journal Editors (ICMJE) announced member journals would only consider a paper for publication if the trial was registered before recruitment began, and included information on the outcomes to be measured, sample sizes and funding sources.
This would make it harder for vested interests to hide unfavourable trials or report on outcomes other than those the trial had originally been set up to study.
It would also help to address the problem of “publication bias”: trials with positive outcomes are far more likely to be published, a distortion that can lead to benefits of a treatment being overestimated.
“If all trials are registered in a public repository at their inception, every trial’s existence is part of the public record and the many stakeholders in clinical research can explore the full range of clinical evidence”, the editors of the participating journals, including the MJA, said in a public statement at the time.
A decade on, how successful has the initiative been?
The number of registered trials has certainly increased dramatically: the largest registry, run by the US National Library of Medicine listed 12 000 trials at the start of 2005. Five years later, the number had reached 83 000 and it is now approaching 200 000.
But problems remain, according to an editorial in The BMJ.
Many journals have still not signed up to the policy and even those that have are sometimes “very generous in allowing exceptions”, the authors write.
They looked at 69 non-complying papers submitted to their own journal over the past 2 years and found authors made a range of excuses for not having prospectively registered their trial.
Senior authors blamed junior staff for the omission. Academic researchers claimed the requirement should not apply to trials without industry funding, or just said they were too busy.
Those excuses won’t wash at The BMJ, which devotes considerable resources to checking not only that trials were registered before they began, but that published reports match details given at the registration stage.
But not all journals are that rigorous or have the resources to carry out those kinds of checks.
A 2013 study of 200 randomly selected journals from the Cochrane Central Register of Controlled Trials database found only 28% explicitly required prospective registration.
Qualitative interviews with a small number of editors indicated one reason for not requiring registration was they feared missing out on interesting articles that would then be picked up by competitors.
Another recent study found that even journals that had endorsed the requirement for prospective registration did not consistently apply the rule.
Only 51% of 747 journals studied included the requirement in their instructions to authors. In a follow-up survey, just 18% said they checked to see if submitted papers matched their registration details.
Two-thirds said they would consider retrospectively registered studies for publication, a concession that pretty much undermines the whole system.
So perhaps it’s not surprising, though more than a little depressing, that another study found that discrepancies between original registration details and those in a published paper had no significant effect on its chance of being published.
The move towards requiring all trials to be registered before they started was an immensely positive one — “the single most valuable tool we have to ensure unbiased reporting”, as the BMJ editorialists put it — but it seems we’re not there yet.
Jane McCredie is a Sydney-based science and medicine writer.
Not only has there been a problem with drug manufacturers withholding data in articles submitted for publication; there is also a problem of failure to publish clinical trials of drug whose results showed harms. Even though a drug may be approved by the FDA, not all the clinical trials of that drug may have showed positive results; however papers on those trials will likely not be submitted for publication. For example, a 2008 paper listed the discrepancy between the number of published articles with positive results and the number of clinical trials that had been submitted to the FDA (1). For paroxetine, just one of the drugs addressed in the 2008 paper, there were 8 journal articles about paroxetine trials which showed positive results, and 2 journal articles about paroxetine trials which showed negative results. However, the manufacturers of paroxetine had submitted data from 16 clinical trials to the FDA. Nine of these 16 trials showed negative results. Were the negative trials ever published? There are also problems with clinical trials undertaken for another use for an already approved drug. If such a trial shows not-previously known harms of that drug in respect of the already approved use, and if that new use is not approved by the FDA, the newly found harms data do not see the light of day. The clinical trials associated with the unapproved use are not released by the FDA because they are treated as commercial-in-confidence so that no other company can use those data to develop a new drug.
On 10 September 2016, the US Department of Health and Human Services released a document Clinical Trials Registration and Results Information Submission – Final Rule (2, 3), which will become effective on 18 January 2017, with a 90-day window for compliance. The Final Rule seeks to redress each of the problems discussed above by expanding the legal requirements relating to the mandatory submission of clinical trial results to the ClinicalTrials.gov database. The Final Rule specifies exactly what must be in those submissions of results so as “to provide more complete information for those who use evidence from clinical trials to inform medical and other decisions” (p.10). A new requirement is for the uploading of the full protocol and statistical analysis plan for each clinical trial (p.81-83). Comment was made that journal editors, reviewers, and readers should have the opportunity to verify the a priori or post hoc nature of trial outcomes. There remains the question about whether the results submitted to ClinicalTrial.gov match those in the FDA database. The petition below seeks to redress this.
The Final Rule includes reporting requirements that will ensure “that each individual’s participation in a trial is appropriately respected” and becomes publicly known (p.23).
There is a petition (4) underway to the US Congress requiring co-ordination between the US National Institutes of Health (which hosts ClinicalTrials.gov) and the FDA, to ensure that what is reported to ClinicalTrials.gov is the same as the information submitted to the FDA. The petition notes that the FDA does not monitor the information placed on ClinicalTrials.gov for fidelity to the FDA’s database, and this “has permitted serious misrepresentations of clinical trials reporting in peer reviewed medical journals, to the detriment of public stakeholders”. The petition requests that study protocols and statistical analysis plans be required by the FDA as part of the clinical trial registration process, in addition to their requirement as part of the now-revised ClinicalTrial.gov registration process. The petition requests that FDA/ClinicalTrials.gov monitoring for concordance should be carried out at the time of trial registration as well as when the clinical trial results are uploaded to ClinicalTrials.gov. The petition accepts signatures from any individuals, not necessarily from the US.
(1) http://opentrials.net/2016/08/10/opentrialsfda-unlocking-the-trove-of-clinical-trial-data-in-drugsfda/
(2) https://s3.amazonaws.com/public-inspection.federalregister.gov/2016-22129.pdf
(3) Zarin, Tse et al. have prepared an explanation of the Final Rule, Trial Reporting in ClinicalTrials.gov — The Final Rule, published 17 November 2016 issue of NEJM. See http://www.nejm.org/doi/full/10.1056/NEJMsr1611785#t=article
(4) http://tinyurl.com/hja4ccy with some additional information at https://www.researchgate.net/project/Truth-in-Research-Labeling-Amendments-to-Section-801-of-the-Public-Health-Service-Act?openDialog=collaborators