Posts
Randomized experiments are all the rage in some circles, for example labor economics and especially development economics. The principle is simple: create some intervention in some market, randomly draw a group of economic agents that has access to the intervention, leave the others out, compare outcomes. In all that, you hope the behavior of the non-participants is not affected by the presence of the program to the others. This can be a heroic assumption, for example because market prices may respond for everyone to the intervention.
Pieter Gautier, Paul Muller, Bas van der Klaauw, Michael Rosholm and Michael Svarer show an example where this assumption was violated. The intention was to see how helping Danish unemployed workers find jobs through enhanced guidance was successful. Those who were non-selected had to deal with the job search as usual. In that case, there were some regions where the experiment was not conducted but data still collected. In the two counties where the experiment was conducted, the number of vacancies markedly increased, which logically leads the treated and untreated to have a better shot at finding a job. But, of course, there is also a congestion effect: for the same number of vacancies, if some workers are getting better probabilities for finding jobs, it is getting worse for the others. In the Danish case, overall this turned out to get worse for the non-participants.
Several papers had previously looked at this experiment and concluded the intervention was a great success because participants fared so much better. But the result can of course not be generalized. What if everyone searches more for the same number of vacancies? Nothing changes much, except that vacancies may be filled faster. And what if the number of vacancies increased in those two counties because of the treatment, to the detriment of the other counties? Then applying the program to the whole country should not make a difference. Given the cost of these studies, this is a very disappointing result.
Pieter Gautier, Paul Muller, Bas van der Klaauw, Michael Rosholm and Michael Svarer show an example where this assumption was violated. The intention was to see how helping Danish unemployed workers find jobs through enhanced guidance was successful. Those who were non-selected had to deal with the job search as usual. In that case, there were some regions where the experiment was not conducted but data still collected. In the two counties where the experiment was conducted, the number of vacancies markedly increased, which logically leads the treated and untreated to have a better shot at finding a job. But, of course, there is also a congestion effect: for the same number of vacancies, if some workers are getting better probabilities for finding jobs, it is getting worse for the others. In the Danish case, overall this turned out to get worse for the non-participants.
Several papers had previously looked at this experiment and concluded the intervention was a great success because participants fared so much better. But the result can of course not be generalized. What if everyone searches more for the same number of vacancies? Nothing changes much, except that vacancies may be filled faster. And what if the number of vacancies increased in those two counties because of the treatment, to the detriment of the other counties? Then applying the program to the whole country should not make a difference. Given the cost of these studies, this is a very disappointing result.
1 comment:
This issue is not unique to randomized experiments. The same criticism can be made of observational studies of labour market interventions. In either case, violation of the assumption of no impact on non-participants is a very important issue which is not discussed nearly enough.
Post a Comment