Thursday, December 20, 2007

Impact factors are a fraud

The RePEc blog has a piece that should be an eye opener to everyone relying on impact factors to establish the quality of a journal. The publisher of these impact factors, Thomson Scientific, is unable to provide the data to replicate them! Using the standards of the journals Thomson Scientific is trying to measure, this would be considered fraud and obviously unpublishable.

I find this very unsettling, considering that so much in terms of tenure and promotion decisions in the academic world is relying on this. Imagine the consequences this could have in terms of litigation by those denied tenure... or by universities that were potentially misled.

I have been using these impact factors to decide where to send my manuscripts. I will not use them anymore and rather use the ones that RePEc computes. While they are clearly not comprehensive (but Thomson Scientific's are not either, and probably worse), at least they are openly computed. It looks like all the data is there on the web to replicate, there is just too much of it... but I have been able to replicate my own citation counts.


John Fast said...

Apparently the quantity of one's publications is much more important than their quality, even when that quality is measured by where they are published rather than by the actual ideas or writing in the article.

As my department chair explained to us, "Tenure committees can't read, but they sure can count!"

Of course there are a few exceptions -- places like GMU -- but they are very rare, and easily identified.

Economic Logician said...

I think any reasonable department in a research university looks at quality more than quantity. And particularly in top universities, where faculty seem obsessed with impact factors and citation counts.

Dave said...

Well, EL, not everyone seeking tenure is lucky enough to work in a real research university. ;->=

After watching several of my friends get subjected to something worse than a fraternity hazing, I'm pretty fed up with the process. At a 'teaching' (Carnegie II) university evidently you can't be sure a book, 2-3 chapters, 7-8 articles (1 in a major journal and others in quite good peer reviewed journals) is enough when some dinosaurs are grumpy.