Women and children first? No

The infamous quadruplicate papers of Frey, Savage and Torgler have caused a lot of grief for their multiplicity, yet they yielded a somewhat interesting, yet old result: women and children get priority on maritime disasters, crew are last, and there are some subtle differences among passengers from different nationalities. This result, however, was obtained using a sample of two: the Titanic and the Lusitania. And one also argue that there was some selection bias for the Titanic, as this was a much celebrated inaugural voyage with, let's say, an unusual set of passengers. It could therefore not hurt to increase the sample size.

Mikael Elinder and Oscar Erixson jack up the sample from 2 to 18. And the results are completely reversed. Women are at a distinct disadvantage, crew fare much better than the rest. This is the outcome you would expect from a free-for-all situation where weaker women get pushed aside on the run for the lifeboats. The lesson from this: do not trust a sample of two, even if amplified by four publications.

Bruno Frey: the story that keeps giving

A few weeks ago, I had a post entitled Bruno Frey, the epilogue, thinking that now that the University of Zurich made him a gigantic gift by manipulating the investigation into his behavior and keeping mum, Bruno Frey would have learned to finally shut up. But no, he has still not understood a thing a keeps going on, to the point that was getting daily updates in my email about the latest on him. Let me run a few highlights by you.

As was rumored for some time, the University of Zurich decided not to renew the two-year contract he was entitled to as an eminent retiree. There no official announcement, but it was reported by Olaf Storbeck, then by the Tages Anzeiger, a local newspaper. In the latter, Frey's wife, Margit Osterloh, defends his behavior and confirms that "he will continue working in Warwick", so he has indeed been told to leave the University of Zurich.

This firing then explains the bizarre behavior of Bruno Frey in the preceding days. Indeed, he appeared unusually incoherent in a television show, then wrote an outrageous piece in the same Tages Anzeiger newspaper calling for the defunding of his department (which actually got a major gift that was probably waiting for his departure). His reasoning is that the professors try too hard to publish their research, neglecting working with the media. He also mentions his research is essentially the only relevant one. Never mind that his employer tried very hard to protect him, gave a special status to his students who were exempt from exams, and now that was simply to possible to go on, the university did its best to let the situation quietly disappear to avoid embarrassing him. He answers with a slash-and-burn tactic.

That said, I also got a copy of the report commissioned by the University and looking into his self-plagiarism on the Titanic studies. As mentioned earlier, there is nothing about the many previous cases. View a pdf copy here.

So Bruno Frey will now continue his activities at the University of Warwick, which has a long tradition of hiring prominent retirees to boost its academic ranking. He joined in early 2011, that is right before the Titanic case came up, and early enough to qualify for the next research assessment exercise of the UK universities. But for the University of Warwick to keep any credibility, it ought now to take position on the Bruno Frey case, now that it is his sole employer.

Finally, as the story keep going on, I created a tag just for Bruno Frey.

Bruno Frey: the epilogue?

A little less than a year ago, a controversy erupted about the publishing practices of Bruno Frey and his students. Indeed, they tend to repackage their research and submit it to multiple journals simultaneously (or sometimes successively), without cross-references and without alerting editors to this. This is in clear violation of the submission conditions of most academic journals and even goes against principles Bruno Frey has himself advocated in multiple (of course) publications: there is not enough space for everyone to publish on the one hand, and the pressure to publish leads people to (self-)plagiarize on the other hand. On his homepage, Bruno Frey crows about over 500 or 600 publications, depending on where you look, numbers that are completely surreal for any self-respecting academic economist.

The scheme blew in his face when some editors and some blogs started raising questions when very similar articles about the Titanic, with Benno Torgler and David Savage, appeared in four journals (some say there is even a fifth one in German, but I cannot verify). And the article was not even original, as a similar analysis was done and published 25 years prior and is now standard reading and exercise in statistics courses. Newspapers picked up the story, Frey went into denial but finally confessed to the editor of the Journal of Economic Perspectives, who published correspondence about the case and publicly admonished him for multiple submissions (he do not yet know about the prior literature). But that is only for this case, there are all the other ones. The University of Zurich, from where Bruno Frey recently retired, promised an investigation. That was sometime in the Summer. Since then, nothing.

One could suspect the University would do nothing, as Bruno Frey is the best ranked economist in German-speaking universities. And the prolonged silence clearly seemed to corroborate this. But rumors started circulating in the hallways, rumors that were not encouraging at all. But no evidence from Zurich.

Finally, I got good evidence from a reliable source. And it is indeed not encouraging. The University of Zurich mandated three prominent academics to look into the case. But the mandate was formulated in such a way that only the articles about the Titanic could be analyzed. The experts came to the obvious conclusion that unethical behavior was at play for this case. They could not mention the others and thus the University concluded that this was a one off miss-step. The University gave Frey a verbal admonishment, which does not go on his record, and did not release the report.

But this was not an one-off miss-step. Frey has been banned from the editorial board of Public Choice for a similar case of re-publication. He is by now banned from publishing in at least a half-dozen journals. To make matters worse, he has himself advocated to go against plagiarizers and others that unnecessarily take up valuable publication space. The investigation should have looked at his whole career, like when a scientist is suspected of fabricating data and all his publications are subject to scrutiny. And it is not like the information would be difficult to obtain, it is readily available and people have even compiled it, as documented in the FreyPlag Wiki.

For more about the case, you can read my past blog posts: 30 April 2011, 3 September 2011, 27 September 2011. Also, Olaf Storbeck's Economics Intelligence blog was the one that convinced the University of Zurich to finally (pretend to) act: 4 July 2011, 4 July 2011, 5 July 2011, 6 July 2011, 7 July 2011, 9 July 2011, 20 August 2011, 29 August 2011, 12 September 2011

Bruno Frey's academic utopia

Bruno Frey has fallen into disgrace these days as he has been shown to play dangerous games of self-plagiarism, submissions to multiple journals and hypocrisy in describing the perverse incentives facing researchers. I have argued recently that he is living in a bubble that has now popped.

Apparently not. With this wife Margit Osterloh, he just authored a paper about the impact of rankings on academic publishing. Their argument is that the current emphasis on rankings pushes academicians to privilege publication to science. They want to decouple funding, tenure and promotion from any evaluation metric. Rather, scientist should be carefully selected at their initial appointment and then be given guaranteed funding and only be asked to evaluate themselves. This strikes as a very utopian view of academia, and in particular a view that surprisingly ignores the impact of incentives on motivation. While the Frey and Osterloh utopia may yield in a few cases the expected very innovative researchers willing to take substantial risks along new paths, most would free-ride to a large degree. The best example of this is the French system of "research associates" on the national research foundation, who are appointed right after their doctorate for a lifetime position of researcher with no other significant duties. While this system has yielded some success stories, the research impact of these associates is rather dismal compared to researchers elsewhere who are subject to regular evaluations using publication metrics.

Reading through the paper, I could not resist to see the irony in many of the arguments, where Frey could be really writing about himself. A few quotes:

In academia, examples can be found (e.g. the ‘slicing strategy’) whereby scholars divide their research results into a ‘least publishable unit’ by breaking the results into as many papers as possible in order to enlarge their publication list.

Of course, Bruno Frey and his students are big specialists in slicing.

there is evidence showing that academics with the highest score in publication rankings score only modestly in a ranking based on their contributions in editorial boards (Rost and Frey, 2011).

Note that Frey was kicked off an editorial board for resubmitting a published paper elsewhere.

For example, a broad and international selection of reviewing peers is necessary in order to avoid cronyism.

Frey's colleague Ernst Fehr recently anointed one of his students with the most prestigious prize for an European economists. Frey and Osterloh argue that awards cannot be manipulated, while citations metrics can. I do not think this is true, in fact awards are easier to manipulate because they are determined by small committees. Citations come from the whole research community.

Another example is editors who encourage authors to cite their respective journals in order to raise their impact rankings

Frey requires this for acceptance in the journal he co-edits, Kyklos. At least, the paper does not mention self-plagiarism this time.

Is this paper a mea culpa? It certainly does not read that way. As in his previous writings on the publishing game, he comes across as someone about it all, who tell everyone how things should be while claiming the moral high ground. The paper was completed on August 24, 2011, thus well after all the conflagration about the hypocrisy of Bruno Frey, yet it does not show anything about it. Bruno Frey has not learned a thing. And, do you want to bet that he is submitting this to journal, taking away space that young researchers need to get publications for tenure, as he has argued before? In fact this particular paper would fit much as a blog post than into a journal.

Addendum: And they are doubling up with another paper, entitled "Input Control and Random Choice - Improving the Selection Process for Journal Articles" with the following abstract:

The process by which scholarly papers are selected for publication in a journal is faced with serious problems. The referees rarely agree and often are biased. This paper discusses two alternative measures to evaluate scholars. The first alternative suggests input control. The second one proposes that the referees should decide only whether a paper reaches a minimal level of quality. Within the resulting set, each paper should be chosen randomly. This procedure has advantages but also disadvantages. The more weight that is given to input control and random mechanism, the more likely it is that unconventional and innovative articles are published.

You read it right: instead of peer review, they advocate complete tyranny by editors who randomize over a select set or, alternatively, to decide early who is worthy publishing and then let the chosen few do as they wish. Which is how Bruno Frey and his lackeys have been operating. But at least Frey and Osterloh have had the decency to withdraw that last paper (which is why I print the abstract above).

The Bruno Frey Bubble

About four months ago, I reported about the apparent self-plagiarism by Bruno Frey, David Savage and Benno Torgler. I found the case particularly ironic, as Bruno Frey repeatedly wrote about the fact that the pressure to publish to get tenure can lead to scholar to unethical behavior, and about the lack of space in journals for young scholars to publish the articles needed for tenure.

The case has taken a much larger dimension now, as many more cases of self-plagiarism by Bruno Frey and his students have appeared (see many links in the comments on the post mentioned above). This raises two very important questions: 1) how could such a culture of self-plagiarism arise? 2) How could they get away with it for so long?

To answer the first question, I think we need to put Bruno Frey is the context of the German(-speaking) academic environment. At least in Economics and Business, the typical German professor publishes a lot of rather insignificant articles, in particular book chapters and "Festschrifts." These works are rarely original, and are not expected to be so. There is also a tradition of writing "educational" pieces that explain economic concepts, say the Edgeworth box or voluntary export restraints, for journals targeted towards professionals in industry and government (as well as students). Again, there is nothing original in there, except maybe the way something is explained.

Bruno Frey works within this paradigm. His work lacks creativity in the sense that he recycles a lot of his ideas for multiple publications, often copying extensively his own words. The differences is that he does that at a higher level than his German colleagues, in international journals that are actually read. And many of his original papers are in fact not that original it appears. If we take the Titanic paper as an example, the empirical exercise he performs is routinely done in undergraduate statistics classes with the same dataset. His contribution is pedagogical, he found a good and interesting way to explain something already present in the body of knowledge.

Like a bubble that keeps getting fed by self-fulfilling expectations, Bruno Frey built on his initial success and continued with this strategy and encouraged his students to do the same. And several of them have assembled remarkable portfolios that way. I mentioned that of Benno Torgler in my original post, but there are several others who got into positions that seem beyond the usual reach of a Swiss doctoral program.

There is another way in which this resembles a bubble. The Economics department at the University of Zurich has made considerable efforts over the past decade or so to become a program that can compete with the better departments in the world. It is certainly among the best in Europe. It did so by americanizing itself: dropping to a large extend the rigid chair structure so prevalent in German speaking universities, hiring internationally respect scholars and creating a proper PhD program with courses and exams. Bruno Frey has not followed this trend at all. In fact, he insisted on exempting his students from the course and exam requirements. The Frey group lives in a cocoon apart from the rest of the department, and lives entirely following the role model of Bruno Frey. Call this living in a bubble.

Or a cult. The interaction of Bruno Frey and his students is reminiscent of a prophet and his disciples who follow him everywhere and write down every word he utters. Well, I exaggerate somewhat, but this does definitely not look like a standard interaction between a mentor and his students. It looks like they follow him blindly, and with his everlasting confidence, he makes them follow his example in publishing.

But this bubble is now popping under the assault of widespread scrutiny from editors, the Economics community and an internal investigation at the University of Zurich. The second question of course is how it was possible for Bruno Frey to act so unethically for so long (he is 70). It appears that he has been caught in the past, but it never became public, or at least explicitly. For example, he has been booted out of an editorial board, but there was no mention of why, his name just disappeared from the list. Also, the journals he has been publishing in are often not prominent and thus not that well read. In fact, it looks like he targeted them so that the audience would not overlap, including editors and referees (the added bonus of this strategy that it satisfies the goal of increasing the pedagogical reach by reaching very different audiences).

Hiding this unethical may have been helped by the fact that Bruno Frey actually tried to present himself as an expert on publishing ethics in Economics. He has written about the perils of publication pressure and how this can lead to slicing papers into insignificant bits, to self-plagiarizing and other unethical behavior. He has complained loudly about the ranking craze which he has been so adept to exploit, both with his self-plagiarism and by requiring authors to cite other works in Kyklos to increase its impact factor. While he is certainly not the only editor to do so, it is ironic that he openly campaigned against such practices. Bruno Frey abused the moral high ground in which he pictured himself.

But as every lie that grows too big over time, this is unsustainable. And it will be less likely to happen in the future with initiatives like this one. Making this unethical behavior more visible will prevent it.

That said, self-plagiarism is not limited to the Bruno Frey group or German speaking economists. I will discuss soon another case that I find particularly enraging.

On the ethics of research cloning

Even though the Journal of Economic Perspectives recently went open access, a move the American Economic Association should be applauded for, I am still receiving physical copies. It is a nice journal to read while lounging in the garden or on a plane ride. The last issue has as usual a good set of interesting articles, including one I had reported on earlier when it was still a working paper. But while checking what I had said about it, I noticed something rather odd: the paper I discussed was ultimately published in the Journal of Economic Behavior and Organization. I had to investigate.

The two papers are by Bruno Frey, David Savage and Benno Torgler. They both report on the sinking of the Titanic and discuss the characteristics of the passengers who survived versus those who perished. Both papers come to the same conclusions. The texts are different, though, and the published regressions are slightly different, with no explanation why, because there is no reference to the other paper. One has therefore to read in much detail to understand what the contribution of each paper is, if there is any.

All this is very fishy. It really looks like the authors are playing games here, trying to get multiple publications out of the same work. They do not mention the other work to fool editors and referees into thinking these are original contributions, as required for any submission to those journals. They tweak the results and rewrite the text so that they cannot be accused of blatant self-plagiarism. This is unethical behavior, but it is not unheard of in the profession.

But like a late-night infomercial, there is a bonus. Looking at the author's CVs, I notice that they have a third publication with the same topic and results, in the Proceedings of the National Academy of Sciences. Bruno Frey has also published two short pieces in German in magazines prior to the academic publications: 1, 2, both pdf.

Now, who are the authors? David Savage is a PhD student at Queensland University of Technology. He must have been following orders of the more senior authors, either without realizing their unethical behavior or watching in horror and not being able to do something about it. Let us give him the benefit of the doubt. His adviser is Benno Torgler, who has already an impressive track record for someone whose first refereed publication was in 2002. His RePEc profile lists 105 working papers and 52 journal articles. Looking at the published works, it seems to like to revisit previous papers by adding new twists to them. Nothing wrong with that, but it may explain why there is no major hit in the publications. There is simply too much slicing and no single slice is a major contribution worth a good publication. But early in his career, he published a series of articles on tax morale using the World Values Survey. Using the same data and the same methodology, he managed to publish several articles whose distinguishing feature is only that they look at a different set of countries: Asia, transition countries, Canada, Latin America, and possibly more. While I must confess that I have not read the papers in detail, there is simply too much material, and Benno Torgler may be innocent, I still find these patterns very disturbing.

It took me some time to figure out where Benno Torgler earned his doctorate. It is at the University of Basel, under the supervision of René Frey (Basel) and Bruno Frey (Zurich), who are brothers, after undergraduate studies at the University of Zurich. Which bring us to Bruno Frey. He is a researcher of international recognition, mostly for his work on welfare economics, happiness research, and critiques of fundamental assumptions in economic models. He credits himself with over 600 published articles and books, an astounding number in Economics. Of course, if this number comes about by slicing papers or republishing known results as described above, this number is less surprising. Looking at his list of major articles, one can surely suspect something is not quite right. I do not have the time (or the will) to go all of this, but there is indeed a lot of rehashing the same themes, which is OK when one uses new data sets or new approaches. But seeing those quantities, that seem unlikely.

Another aspect that I find disturbing in Bruno Frey's record is that his recent work has been railing against the tendency of academics (and especially their administrators and grant makers) to look for quantifiable evidence of their productivity, what he calls "evaluitis." He writes against the pressure to publish and the prominence of rankings of research output. I have reported about some of this writing myself (1, 2, 3). But again he seems to be repeating himself a lot, even in published articles, essentially criticizing a game that he seems to be excelling at. Either he is sarcastic or hypocritical, I cannot decide.

I realize the accusations I am making here can have severe consequences. But I am only accusing, not condemning. I leave the reader the opportunity to make her own opinion, as I have linked to plenty of evidence. I hope to be proven wrong, that these three individuals are indeed extremely innovative and productive. But from what I have seen so far, my prejudice is strongly negative in this regard.

Update (Sunday): I have been alerted that there is a fourth publication about the same Titanic study, in Rationality and Study.

Further update: A follow-up post.

Academia vs. university

I have recently opined on the status of academia, and I am more pessimistic about the survival of the current model in the United States than in Europe, in particular with respect to their research and teaching missions. I find it interesting to see that Bruno Frey seems to share some of this pessimism. But for somewhat different reasons.

Frey longs for the old model where the university is an untouchable and well-funded institution where academics spend their day thinking about research with little regard to what happens around them, While I think some ultra-talented researchers should have such privileges, this does not apply to the very vast majority of current university faculty whose research contributions are very marginal at best. While it important that they carry out research to get their teaching current, research should not be a resource draining focus.

Frey is afraid that his utopia of university is falling apart not because of market forces and costs, because of internal governance. He complains that research is nowadays too specialized. I am afraid this is unavoidable at the frontier, yet in Economics it is probably the least so as many publish in very different fields (and even outside of Economics). I agree with him that there is too much pressure to publish in mid-range universities where teaching should be the focus. He also claims that academia is rife with fraud, something I cannot testify to, but maybe I am naive. He finds also that academia is withering because universities admit too many students (I fully agree) and because people do not need to network within a physical university location to conduct research.

But the main reason for which academia is falling apart is, according to Frey, the emphasis ("mania") on rankings. While I agree that rankings are abused, they have a major justification: holding the university and its members accountable. Every university can (and does) claim it has the best teachers and researchers, but rankings put some realism in this. But what surprises me most in Frey's claim is that he is himself ranking obsessed to the point of venturing in unethical behavior. Indeed, he is managing editor (with his brother) of Kyklos, an old journal that has considerably lost in reputation. To improve its impact factor, he requires that accepted papers cite other articles from Kyklos...

How to price museum visits

So you come across a museum and wonder whether it is worth visiting. The entrance price is rather steep, and given the uncertainty of your future enjoyment, you decide to pass on the opportunity and walk on. Could the museum have found a way to still attract you? One way would be to give you free entry, but this would oviously lead to loss of revenue and overuse of the museum.

Bruno Frey and Lasse Steiner claim it can attract casual visitors by asking for an entrance fee that depends on the time spent in the museum. This would allow for a cheap "sneak peek," and true lovers would pay much closer to their marginal utility. This is more efficient than exit donations, as those are voluntary and thus lead to loss of revenue and overuse of the museum. Keeping in mind that a museum's mission is often to spread knowledge and enjoyment, this scheme is much more likely to attract visitors with lower marginal utility and possibly expose them to arts or sciences they may learn to appreciate. Instead, museums typically emphasize club memberships, who cater to visitors who would have come anyway and offer them deep discounts that bring revenue loss.

Catch-22 for publish-and-perish

When it comes to publication, economists set incredibly high standards. Compared to other fields, rejection rates are higher, editors and especially referees have a much higher impact, papers go through multiple revisions within the same submission (and may still be rejected), papers are expected to be self-contained pieces, and everything needs to be perfect, including grammar. All this leads to the well-lamented huge publication delays in Economics. Yet, publication in the top journals is still a requirement for tenure in any department that wishes to garner any respect, and it is just not possible for top journals to publish all the research fit for tenure.

Bruno Frey calls this the "Publication Impossibility Theorem System" (PITS): there just not enough slots in those journals, especially once one considers that 80% of the articles in the top five journals are written by faculty based in the US. Frey claims that this makes it impossible for non-Americans. Whether there is a US bias remains to be established, but even for Americans, the bottleneck is clearly present.

That said, this brings up more generally the question of publication expectations for tenure and promotion, and the journal culture in Economics. Over the last two decades, a large number of journals have been created, partly because commercial publishers saw opportunities, but mostly because the top, society driven journals failed to realize that the publishing needs of the profession have increased. It is only very recently that the American Economic association has realized that it missed the boat and created its four new American Economic Journals. The econometric Society as well announced the creation of two "field journals", Empirical Economics and the preexisting Theoretical Economics.

This should provide only little relief, as these add-on journals are unlikely to be considered top-journals. In fact they seem to turn into junk mail lying around everywhere in department mail rooms. What is really needed is to write shorter papers, straight to the point with less emphasis on the referees. More papers will then be published faster, and let citations then do the measurement of quality. The latter are a very imperfect measure nowadays, because the excessive lengths of papers allows to cite each and everyone. A shorter paper will have to be more selective and restrict itself on those that it really builds upon.

Fairness, culture and selfish American men

Economic theory mostly assumes that individuals are self-interested, yet there is plenty of evidence that they also value fairness and incorporate the utility of others, including people they are not related to. So they may not be that self-interested or, put it in another way, their utility function has other arguments that pertain to other individuals or to aggregate measures. Sociologists and psychologists would argue here that at least part of this interest in others comes from education or societal pressure. Many studies have highlighted that some societies are more caring, and others more greedy. Much of this is based on surveys.

Economists do generally not like surveys because they do not reflect actual decisions. But controlled experiments or good data are very hard to come by. Bruno Frey, David Savage and Benno Torgler found an interesting data set: the passengers of the Titanic. There is a well established societal norm that women and children have priority in situation of life and death where a future reciprocity is not expected. It this norm really applied?

Take as an example the contrast between British and American customs: In Britain, people queue for everything and apply strictly the norm of :first come first served". In the US, the price is much more used as a selection mechanism, thus wealth matters more. In the context of the Titanic, this would mean: among Brits, a larger proportion of women and children than men should have survived; among Americans, the survivors should be more frequent in first class than third class. According to this analysis by John Henderson, this hypothesis would be correct: the survival rate in third class for British women is 47%, British children 35% and British men 12%. Americans in first class survived at a rate of 67%, 47% in second class and 28% in third class. Interestingly, 58% of Americans survived against of 33% Brits. Did Americans push their way to the lifeboats?

But the analysis is not that simple: there can be composition effects: American may have been proportionally more numerous is higher classes or with more females. Few staff survived, and they may have been disproportionately British. Thus, simple averages are not enough, some proper regression analysis is required using a variety of controls. This is what Frey, Savage and Torgler do and they confirm that women and children had higher survival rates, thus societal norms seem to have largely prevailed. Passengers from higher classes also had a higher survival probability, but this result is likely tainted by the fact that 1) they were better informed as early on few believed the Titanic would sink, 2) most boats were on the first class deck.

And even with all those controls, the British are more likely to die and the Americans are those most likely to survive. But this distinction is not visible among women. Ah, those selfish American men.