Credit: MAJO XERIDAT

In February 1987, just months after the discovery of high-temperature superconductivity in materials known as cuprates, labs worldwide were locked in heated competition to identify new members of the family. That month, in two papers submitted to Physical Review Letters, researchers led by Paul Chu of the University of Houston and Maw-Kuen Wu of the University of Alabama in Birmingham described a new superconducting material as “ytterbium barium copper oxide”. But with the manuscripts on the verge of publication, the authors changed the name of the material: the element “ytterbium” became “yttrium”.

Many materials scientists suspect the error was deliberate, designed to protect the discovery during the peer-review process. Chu originally blamed a typographical error, and today refuses to comment. But it is clear that he and Wu had reason to worry. “There is no doubt the manuscript was leaked during peer review,” says Paul Grant, a superconductivity researcher at the Electric Power Research Institute in Palo Alto, California, who has followed the story closely. While the paper was under review, says Grant, researchers from at least two other groups contacted Chu to tell him that the material with the erroneous element did not work as a superconductor.

Paul Chu was contacted about his paper on superconductors while it was still under review. Credit: UNIV. HOUSTON

Speak to researchers in certain fields, and tales about abuses of peer review start creeping from the woodwork. Some scientists complain of manuscripts being stalled in review until similar findings emerge in another journal. Was the reviewer an author of the second paper who failed to declare this conflict? Did he or she deliberately delay the manuscript so as to publish first? Had the competing paper already been submitted to another journal, or could the work have been conducted, submitted and published after reading the manuscript? Other researchers worry about reviewers who are consultants to companies and who could pass on information that should remain confidential. Might such instances help companies win the race to market new discoveries?

Articles of faith

These issues will be debated this week, as journal editors, scientists and other interested parties meet in Barcelona for the Fourth International Congress on Peer Review in Biomedical Publication. The conference — held roughly every four years — is the brainchild of Drummond Rennie, a professor of medicine at the University of California, San Francisco, and a deputy editor of the Journal of the American Medical Association.

Rennie is a strong supporter of the value of peer review, and Nature's interviews with other journal editors and researchers reveal that most agree with him. By and large, they argue, peer review is a mutually beneficial system that provides effective quality control in publication and grant awards. But although instances of abuse may be rare, the process is not problem-free.

No doubt: Paul Grant (right) is certain that leaks occur, particularly in highly competitive fields. Credit: DETROIT EDISON

In some areas, such as geology — where important findings often depend on years of meticulous fieldwork — complaints are few and far between. But the potential for abuse is greater in fast-moving fields such as molecular biology, in which it is sometimes possible to read a manuscript and replicate the work in a matter of days. Where problems arise, the intense drive of individual scientists to further their standing is most often to blame. But commercial and, in rare cases, political pressures (see 'Box 1 Igniting an unholy row') can also come into play.

Reviewers who directly misuse the information contained in manuscripts or grant applications are only part of the problem. Many reviewers admit to discussing papers or grant applications that they are sent for review with their colleagues, which increases the chance that someone might abuse that privileged information.

In late 1996, for instance, molecular biologist Carolyn Price, then at the University of Nebraska at Lincoln, was reading through grant applications in molecular cytology for the National Institutes of Health (NIH) when one application looked suspiciously familiar. She pulled out one of her own grant applications, which she had submitted earlier that year to a state funding agency, and found that sections had been plagiarized.

“It was bizarre — a twilight-zone experience,” says Price, now at the University of Cincinnati in Ohio. A subsequent probe revealed that a Los Angeles-based reviewer examining Price's application had shared the document in confidence with biochemist Ashraf Imam at the University of Southern California (USC). The Office of Research Integrity, the watchdog body that investigates allegations of misconduct surrounding NIH grants, concluded that Imam had plagiarized material from Price's application. In 1998, he agreed to be debarred from receiving NIH funds for three years. Imam now works on gene therapy at the Huntington Medical Research Institutes in Pasadena, California. Clive Taylor, professor of pathology at USC and Imam's boss at the time, attributes the event to “the enormous pressures on non-tenured faculty”.

Drummond Rennie: striving to improve a valued system. Credit: UNIV. CALIFORNIA, SAN FRANCISCO

John Hardy, a neuroscientist who is this month moving from the Mayo Clinic in Jacksonville, Florida, to the National Institute on Aging in Bethesda, Maryland, says that he sometimes discusses manuscripts sent to him for review with lab colleagues — with the proviso that they cannot reveal the material to anyone else or use it for a publication or grant application. Hardy argues that this is common among researchers in his field, and believes the issue needs to be recognized and debated. “I don't believe people when they tell me they don't talk to people about manuscripts they receive for review,” he says.

But some researchers would like to see the practice stamped out. “It is terrible,” says Randy Schekman, a molecular biologist at the University of California, Berkeley, and an editor of the Journal of Cell Biology. “I know it occurs, but I hope not widely. I think it is a breach of confidentiality that really compromises the process,” he says.

Despite Schekman's reservations, most scientists questioned by Nature argue that limited discussion of papers is acceptable and should improve the quality of review. Some journals do issue clear guidelines, however. Nature, for example, asks its reviewers not to discuss confidential manuscripts with colleagues unless this is necessary to assess its merits — and where this is the case to provide the names of those involved.

Commercial breaks

When a paper contains commercially sensitive information, breaches of confidentiality are a particular concern. Many researchers act as consultants to companies, and once papers under review start to be discussed, there is always the chance that sensitive information will reach the ears of someone with a commercial interest in it. Many researchers are certain that this happens, but such cases are notoriously hard to prove.

In 1991, a genetics paper submitted to Nature was at the heart of one such murky incident. Written by a group at a British university, the paper contained details of a discovery likely to be of interest to many biotechnology companies. One author of the paper is convinced that a company knew of the discovery before the paper was published.

His suspicions were first aroused about two weeks before publication, when he learned that a Newsweek reporter had been asking other researchers in the field about the paper. Then, just two days after publication, the group received a visit from a senior representative of a US-based biotechnology firm who wanted to license the rights to develop new therapies based on the discovery. But despite the author's suspicions of a breach of confidentiality, the facts of the case provide no 'smoking gun'.

Nature's interviews also reveal that it is often difficult to distinguish between genuine complaints and paranoia. Many editors can relate tales of anguished authors convinced that a reviewer had abused the process. Such arguments often surround negative reports perceived by the author to have come from a particular rival intent on blocking the paper's publication — but who had not, in fact, been sent the manuscript. “An author will suggest who to send the manuscript to,” says Schekman. The journal may do so, but if that reviewer delivers a negative report, the author may assume wrongly that it came from his or her rival. “Then they howl, because they don't like the response,” Schekman says.

But despite the problems thrown up by peer review, no serious alternative has yet been proposed. “It is easy to say the system is flawed; it is harder to say how to improve it,” says Ronald McKay, a stem-cell researcher at the National Institute of Neurological Disorders and Stroke in Bethesda, Maryland.

One tweak to the process — asking reviewers to sign their reviews — has been experimented with. The idea is that, if reviewers are obliged to identify themselves, it will improve transparency and discourage anyone who might be tempted to abuse the process under the cloak of anonymity. Rennie is a particular enthusiast for this approach. “This is the only credible, worthwhile, transparent and honest system,” he says. “I've made that passionate plea, but the majority hasn't gone along with it.”

Indeed, when the idea was first mooted many years ago, some editors were openly hostile. In the 1960s, when David Sackett, now professor emeritus at McMaster University in Hamilton, Ontario, started signing his reviews, he was removed from the review boards of journals such as The New England Journal of Medicine.

Attitudes have softened since then, but some editors remain convinced that the practice has its problems. “I personally discourage it,” says Stephen Lisberger, a neurophysiologist at the University of California, San Francisco, and an editor for the Journal of Neuroscience. “It encourages an author to bypass the editorial process and correspond with the reviewer. Then you get negotiation and compromise between the author and reviewer.” To minimize such problems, some journals — including Nature — only reveal the names of reviewers who want to be identified in the final round of review.

Identity crisis

Lisberger also recalls an experiment in the 1980s at the Journal of Neurophysiology in which reviewers were encouraged to sign their reviews. “We found reviewers signed their positive reviews, but not their negative ones,” he says. “It didn't change the backbiting, or complaints that so-and-so was blocking publication.” The journal eventually dropped the policy.

Other scientists and editors argue that the Internet has helped to reduce abuses. One common fear is that reviewers may be deliberately slow in responding if they wish to delay a rival's paper. But instantaneous communication by e-mail has greatly reduced the scope for reviewers to blame non-delivery of manuscripts and reports for such tardiness. Also, the availability of literature online means that researchers who plagiarize information gained during peer review are more likely to be found out. “You can scan journals so much quicker now,” says Ralph Yount, a biochemist at Washington State University in Pullman and former president of the Federation of American Societies for Experimental Biology. “People can't steal things and publish in an obscure journal somewhere.”

Such abuses may be relatively rare, but they are a serious business for the researchers on the receiving end. So what happens in such instances? As the Imam case shows, blatant abuses such as plagiarism of grant applications can lead to tough sanctions. But journal editors say they are rarely in a position to act as judge and jury when suspicions arise. Often, the facts are far from clear. For example, a reviewer might agree to review a paper on being sent its abstract, and then sit on the full manuscript for several weeks before returning it, declining to act as a reviewer because he or she has a competing manuscript under review with another publication. Does that represent a cynical attempt to delay a rival's publication? Or is it an innocent slip-up by a busy scientist?

Where it seems that a reviewer has abused the process, the most common course of action is to challenge the individual concerned, and if he or she cannot convince the editor of their innocence, for the journal to cease using that person as a reviewer. Science's editor-in-chief, Donald Kennedy, says that, in particularly egregious cases, a journal might refuse to consider an offender's own manuscripts for publication.

Penalty clause

Other editors have, on occasion, referred accusations of misconduct during peer review to the accused individual's host institution. Nicholas Cozzarelli, professor of biochemistry and molecular biology at the University of California, Berkeley, and editor-in-chief of the Proceedings of the National Academy of Sciences, says he has found this to be the most effective policy. But he says that taking such action requires an accuser to make a formal complaint — which some aggrieved manuscript authors are loath to do for fear of attracting a lawsuit from the accused.

Philip Campbell, editor of Nature, says that allegations of abuse during peer review are dealt with on a case-by-case basis. In addition to the sanctions listed above, Nature may also write an article about clear instances of misconduct. He acknowledges that it is rare that one can be sure enough to take the stronger sanctions. “But some employers, in becoming tougher against misconduct, are encouraging us to report even suspicion of bad behaviour in peer review. We'd be very cautious about that.”

Many editors also argue that procedures to minimize and identify abuses are more important than sanctions against those who offend. Journals usually demand that reviewers declare any conflicts of interest that might influence their ability to provide an unbiased assessment of a paper. Simple procedures such as acceding to authors' requests not to send manuscripts to particular competitors also help to avoid problems. And if manuscripts are routinely sent to two or three different reviewers, suspicions should soon arise if one reviewer raises spurious objections to a manuscript in an attempt to block a rival's publication.

The Barcelona meeting may well throw up other suggestions for improving the system. But despite its flaws, peer review will continue to form the bedrock of the scientific enterprise. For every allegation of abuse, there are countless more papers that are improved by helpful suggestions made by reviewers. “I've seen the amazingly wonderful things peer review does for authors,” says Rennie. “I would hate to give it up.”

Fourth International Congress on Peer Review in Biomedical Publication

→ http://www.ama-assn.org/public/peer/peerhome.htm