Monday, June 2, 2008

The peer review system is broken

No, I did not receive a rejection from a journal this morning. I still want to argue that the peer review system is fundamentally broken in Economics. And Economists should really know better.

Referees write reports that are anonymous to everyone but the editor. They can make claims that are not open to scrutinization. They can abuse the author by asking her to modify paper to suit their agenda (cite them, in particular). They are also not held accountable if they do a lousy job, take forever, or do not care.

Of course, the identity of referees is open to editors. But in many cases, the editors have no way to identify whether referees do a good job, the latter are supposed to be the experts, after all. This is a typical agency problem, where often the principal has little idea of what is going on. The consequence is that journal publications correlate little with quality, and in particular that good works get rejected for flimsy reasons.

The only way out I see is to make referee names public. To some extend this is already done for some institutional working papers, for example at the IMF where the person authorizing the publication is listed on the paper. But this should also apply to rejections. Journals typically acknowledge their referees in one issue a year, but it is difficult to judge them on this.

Rating agencies for financial products are under scrutiny these days. Referees in Economics should as well.

3 comments:

Independent Accountant said...

EL:
Welcome aboard. I have derided peer review in the CPA profession for over 30 years! Why would I expect it to be any better in your business? I don't. I see peer review as a fraud.

Independent Accountant said...

EL:
Gary North had a post over at www.lewrockwell.com, on 3 June 2008 I sympathize with. You may to. Be warned, I am a doctrinaire Austrian, despire having a Chiacgo MBA. Repeal the Federal Reserve Act! Storm the Bastille!

VIlfredo said...

I agree, down with the tyrany of the faceless referees!

Note that blogs like this one have, potentially, a role to play in validating research. But Economic Logician is faceless as well...