Monday, July 03, 2006

For Science's Gatekeepers, a Credibility Gap -- NY Times

Here's the letter I wrote in response to this article (which was never published):

I read with great interest the recent Science Times about problem's with the
gatekeepers in science -- journal editors and the referees they commission in
the peer-review process. This article exaggerates the problem. In my opinion,
most practicing scientists do not believe that just because something is
published in a journal, it means that it is correct. Moreover, the supposed
"quality" of the journal does not correlate with correctness of its results but
rather simply with its perceived newsworthiness. In contrast, most scientists
feel that accepted results have to pass the test of replication over time in
different laboratories and in different people's hands. That is, an idea has to
be successful in the academic marketplace, just as an everyday "meme" achieves
success in the real marketplace. Furthermore, the suggestion in the article that
we increase the stringency of the editorial process has a number of potential
negative consequences. First of all, it will invariably slow down the
publication of ideas, both good and bad. Second of all, if we increase the
severity of the review process, we potentially increase the likelihood that
papers will be rejected for a variety of non-scientific reasons, e.g. they are
not fashionable at the moment or a referee is a competitor of the author and
does not want a particular result published. What we really need to do is to
encourage scientists to constructively replicate and verify each other's work
and to report both negative and positive findings with regard to this. The
essential problem is one does not get much credit in science for doing this --
all the accolades go to the person who came up with the idea first. Thus, rather
than emphasize gatekeepers we need to provide proper incentives for careful
public replication of findings.

For Science's Gatekeepers, a Credibility Gap
Published: May 2, 2006
Recent disclosures of fraudulent or flawed studies in medical and scientific journals have called into question as never before the merits of their peer-review system....

ooo[clip]ooo ooo[bioinfo]ooo ooo[L2E]ooo