By Michael Kranish, Globe Staff, August 15, 2005

WASHINGTON -- They are two of the most widely publicized pieces of medical research in recent years: Reports in prestigious journals declared that women who underwent hormone replacement therapy, and people who ingested large amounts of Vitamin E, had relatively low rates of heart disease.

Each study was vetted by peer review, the basic process for checking medical research, in which other researchers judge whether papers meet scientific standards.

Now, after a study that sent reverberations through the medical profession by finding that almost one-third of top research articles have been either contradicted or seriously questioned, some specialists are calling for radical changes in the system.

But after research contradicted those studies -- frustrating anyone who had followed their recommendations -- some specialists began looking at whether peer review had failed to identify serious flaws in the research.

But the specialists found that it was almost impossible to discover what had happened in the vetting process, since peer reviewers are unpaid, anonymous, and unaccountable. Moreover, their reviews are kept confidential, making it impossible to know the parameters of the reviews.

Now, after a study that sent reverberations through the medical profession by finding that almost one-third of top research articles have been either contradicted or seriously questioned, some specialists are calling for radical changes in the system.

In advance of a world congress on peer review next month in Chicago, these specialists are suggesting that reviewers drop their anonymity and allow comments to be published. Some are proposing that peer reviewers be paid to ensure a more even quality of review and analysis among all journals.

Dr. Drummond Rennie, who relies on review as deputy editor of JAMA, the Journal of the American Medical Association, said of the process, ''The more we look into it, the harder it is to prove whether it does good or bad."

Rennie has called for greater study of whether peer review improves research, and he has a personal policy of disclosing his name when he reviews articles.

''It would be lovely to start anew and to set up a trial of peer review against no peer review," Rennie said. ''But no journal is willing to risk it."

Rennie's journal published the study, which said that subsequent research had found that almost one-third of the top papers that appeared in top journals over a 13-year period from 1990 to 2003, had been either contradicted or found to have potentially exaggerated results. All the articles had undergone vigorous peer review, leading to questions about whether problems should have been caught by reviewers.

The author of that study, Dr. John Ioannidis, an adjunct professor at the Tufts University School of Medicine, said that flaws in the system were not solely responsible for the problems with the initial studies, but he said that they may be ''part of the puzzle" that should be examined to improve research.

Ioannidis has proposed making peer reviews public so that ''one could see whether someone said, 'This is a great study, publish it,' or whether there was constructive scientific thinking, comment and criticism." He noted that he could not examine any peer reviews, including those for the hormone replacement and Vitamin E studies, because of the confidentiality surrounding peer review.


Next >>

Like What You Just Read? Try These Similar Links...

·  Many Researchers Break the Rules - HealthDay News ·  How To Read Scientific Studies in the Media: Sorting out Scientific Studies - PBS.org ·  Mercury Rising: The toxic metal isn't just in seafood. It's showing up everywhere--and it's more dangerous than you think