Reviewing Peer Review

In recent years, the practice of "peer review" seems to have been elevated to a higher status than ever in public discussion of scientific results - particularly in climate science. The pro-human-caused-warming people always ballyhooed the peer-reviewed status of their articles.

But today I came across an article claiming that actual studies of "peer review", in medicine, have found that it doesn't work as hoped:

If peer review was a drug it would never be allowed onto the market,' says Drummond Rennie, deputy editor of the Journal Of the American Medical Association and intellectual father of the international congresses of peer review that have been held every four years since 1989. Peer review would not get onto the market because we have no convincing evidence of its benefits but a lot of evidence of its flaws.

Yet, to my continuing surprise, almost no scientists know anything about the evidence on peer review. It is a process that is central to science - deciding which grant proposals will be funded, which papers will be published, who will be promoted, and who will receive a Nobel prize. We might thus expect that scientists, people who are trained to believe nothing until presented with evidence, would want to know all the evidence available on this important process. Yet not only do scientists know little about the evidence on peer review but most continue to believe in peer review, thinking it essential for the progress of science. Ironically, a faith based rather than an evidence based process lies at the heart of science.

Exactly how central is peer review is to science? Couldn't you do science if you were the last human alive? You wouldn't be able to check with your colleagues, but you could still run experiments and check your theory against the evidence.

Peer review
appears to do
quite a bit less
than most might guess.