Jane McCredie: Accentuate the negative

IF you want to gauge the influence of a particular scientific paper, one thing you might do is check Google Scholar to see how many citations it has.

 

The number you see may offer a rough guide to the importance of the research, but there’s no easy way to know whether it represents widespread acclamation or enraged criticism.

 

After all, Andrew Wakefield’s retracted 1998 paper in The Lancet — the one that fraudulently claimed to have established a link between the measles–mumps–rubella vaccine and autism — boasts an impressive 2192 citations on Google Scholar, but few researchers would want to emulate that particular kind of notoriety.

 

Wakefield aside, negative citations may not always be a bad thing. In fact, they might actually reflect how engaged the research community is with a study and its findings.

 

North American researchers have analysed a phenomenal 762 355 citations from 15 731 papers, published in the Journal of Immunology over the decade to 2007, to examine the incidence and role of negative citations.

 

That’s research on a scale that would have been impossible without recent developments in natural language processing and the ability to parse and analyse large bodies of text, the researchers write in Proceedings of the National Academy of Sciences.

 

The proportion of citations classified as negative was low (about 2.4%), but just over 7% of the cited papers were found to have received at least one negative citation, “a nontrivial number”, as the researchers put it.

 

About 84% of the negative citations appeared in the results and discussion sections of papers, suggesting the criticism focused on findings rather than methods or theories.

 

Interestingly, these researchers found studies receiving negative citations were generally of high quality and prominent, receiving a higher median number of citations than papers that were never cited negatively.

 

This might be, they suggest, because “as scientists pay more attention to a study (potentially also because of its novelty and quality), they are also more likely to provide criticisms, extensions, and qualifications to it. Negative citations can be therefore seen as a way to track where scientists place attention at a certain time in the field”.

 

They were also more likely to come from scientists working in a closely related field, prompting the authors to suggest negative citations might be “one of the ways in which scientists debate and make progress in their field of research”.

 

Negative citations were, however, less likely to come from researchers who were geographically close, perhaps suggesting a fear of awkward corridor encounters.

 

This kind of analysis might be just the beginning. These authors argue that classification of different kinds of citations could enhance scientific discourse and help advances in knowledge to be assimilated more quickly.

 

Researchers could be encouraged to add a classification to metadata associated with a citation, which online scholarly directories could then use to improve their search and ranking algorithms, the authors suggest.

 

New information technologies are clearly key to such advances, but the critical examination of others’ work has long been a key element of the scientific process and, as these authors point out, it can be fruitful even when the critic is eventually proved wrong.

 

Back in the 16th century, Danish astronomer Tycho Brahe disputed the emerging Copernican revolution and its argument that the earth revolved around the sun, rather than the other way around.

 

Brahe may have got that wrong, but his observation-based challenge to the new view of the universe pushed other researchers to deepen their investigations, which — paradoxically — ended up strengthening the heliocentric theory.

 

So bring on the negative citations because that, in the end, is how science works.