Jane McCredie: Search for value

IN 2010, global expenditure on life sciences research, most of it biomedical research, was US$240 billion (A$274 billion), according to a paper published in The Lancet earlier this year.

That’s a lot of money in anyone’s books. But are we getting value for it?

Stanford epidemiologist Professor John Ioannidis argues as much as 85% of research resources may be wasted, due largely to conflicts of interest or failures in study design or reporting.

“Interventions to make science less wasteful and more effective could be hugely beneficial to our health, our comfort, and our grasp of truth and could help scientific research more successfully pursue its noble goals”, he writes in PLOS Medicine .

What might those interventions be?

“We must diminish biases, conflicts of interest, and fragmentation of efforts in favor of unbiased, transparent, collaborative research with greater standardization”, Professor Ioannidis writes.

In part, he’s arguing for a cooperative rather than a competitive model of research, one in which research teams would share data, protocols, materials and tools with others.

This would help establish a “replication culture” that would provide stronger validation (or invalidation) of research findings through determining whether they could be reproduced in more than one study.

If replication needs are not considered in designing the research agenda, “some questions are not addressed at all or are addressed by single studies that are never replicated, while others are subjected to multiple unnecessary replications”, Professor Ioannidis writes.

It’s an important point, but achieving the open, collaborative research culture proposed in this paper wouldn’t be easy.

Research teams generally want to be original and, perhaps most importantly, they want to be first. Sharing data and tools with groups they see as rivals may not always come naturally.

On top of that, there’s really not a lot of glory in repeating a study to show that some other team got it right in the first place.

“The current system does not reward replication”, Professor Ioannidis writes. “It often even penalizes people who want to rigorously replicate previous work, and it pushes investigators to claim that their work is highly novel and significant.”

And, while we’re talking about sharing, you have to wonder how likely the pharmaceutical industry would be to join in the love.

In fact, Professor Ioannidis points out a paradox in the industry attitude to that whole question of reproducibility.

When it comes to preclinical research — the stuff often done in universities that helps the industry pinpoint useful drug targets — reproducibility is the companies’ best friend.

They’re not going to invest large amounts of money developing a potential treatment without some pretty robust, and reproducible, preclinical findings.

Get to the clinical trial stage, though, and it’s a completely different picture.

If a single trial of a new drug has achieved stellar results, there’s not a lot of incentive for the manufacturer to conduct a second one just to make sure. They could just end up shooting themselves in the foot.

Providing incentives for these kinds of replicating studies is just one of the strategies Professor Ioannidis suggests to help us achieve a better bang for the billions of bucks spent on medical research each year.

Some of his suggestions, he admits, are “deliberately provocative”, including a complete overhaul of the research “reward system”, providing greater rewards for peer review, for sharing and collaboration, and for contributing to the education of others, while downgrading the value placed on publication.

Improvements in study design and statistical methods are also on the list, as are registration of studies and their results, as well as “containment” of conflicts of interest in sponsors and authors.

I’m not entirely convinced by the 85% figure when it comes to waste of research funds (trawling back through the footnotes led me to suspect it was more a guess than a measurement), but we could certainly aim for a better return on our massive investment in medical research.

The challenge is working out exactly how we might achieve such a goal, something that will, as Professor Ioannidis acknowledges, require a serious research effort of its own.
 

Jane McCredie is a Sydney-based science and medicine writer.