Biases from returnees in experimental economics

Performing economic experiments in a laboratory setting is tricky business. You want to make sure to get an environment that is clear of any uncontrolled influences so that you can concentrate on what you want to test. You want participants to care about outcomes so that results have some meaning. And you want to make sure that each wave of participants gets exactly the same conditions. And what about participant selection?

It is quite difficult to get people to come and participate. To get them through the door, you promise some prize or payment, which you make dependent on performance to entice active participation. Who are you going to get, those that have a low opportunity cost of there time. This obviously biases the sample population. One may try to counteract this by dropping some, but one is usually just happy of getting the required number of people that one cannot afford to lose some.

But wait, you get another population bias for free! As Pablo Guillén and Róbert Veszteg report, returnees are not a random draw either. Rules may prevent them to come back for the same experiment, but they may attend a different experiment. Guillén and Veszteg find that males who have been successful in previous experiments and study social sciences are more likely to return. The experimental literature needs to find a way to control for this bias.