Monday, October 30, 2006

Is peer-review pure pants?

The former editor of the British Medical Journal, Richard Smith, in an admittedly blatent attempt to plug his new book, has published an article panning the peer-review process. Only a fool would claim that peer-review is perfect, but I felt that the article was overly-critical and made claims for peer-review that not even its most fanatical proponents would espouse.

http://commentisfree.guardian.co.uk/richard_smith/2006/10/beware_medical_journals.html

Here is my reply:

It's easy to criticise peer-review, but what are the credible alternatives? Richard Smith ignores the fact that peer-review was never intended to uncover fraudulence: as he points out, it is a system built on trust, and the goal is to establish if the conclusions reached can be justified by the data presented, not to see if the data is fabricated. However, if the research is of significance, then it is probable that attempts to first duplicate then build upon the work will establish its authenticity, as is presumably the case for the fraudulent sudies Richard refers to. As for publication bias, especially in big-pharma funded clinical trials, Ben Goldacre has suggested the wonderfully simple solution of compulsory registration of trials (www.badscience.net).

Regarding the finding of errors by peer-review, Richard Smith makes no reference to the severity of these errors, and makes no reference to the original research, so it is difficult to judge his claims, but two points are worth mentioning: peer-review is intended to pick up serious faults in reasoning and misrepresentation of the established facts, not to find trivial faults such as spelling mistakes and suchlike. Secondly, reviewers are not assumed to be foolproof, hence the reason why reputable journals submit each article to at least two reviewers.

Finally, I would like to point out that scepticism is a quality required of all scientists, and that those who take everything published in journals to be gospel have no place in research.

0 Comments:

Post a Comment

<< Home