Posts
The most important questions economists can address relate to economic development. How can we explain the immense differences in income across the world? Why is human capital so low in developping countries? What can policy do about it? Should policymakers care? But answering all these questions is severely hampered by the abysmal quality of the data. Macroeconomic data is spotty and unreliable, and microeconomic data is largely inexistent. The answer to this problem was for researchers to generate their own data.
Thus were born randomization studies, whereby some region was region was subject to an economic experiment. Randomly seleced people or villages are given some sort of incentive, and others not, and the impact of the intervention is studied. This procedure has now become extremely fashionable in the development economics community, where it is now a must to be working "in the field" gathering data. This approach has, however, become increasingly questioned, for several reasons.
First, randomization studies are terribly expensive. There is an increasing sentiment that these resources could be better used, both in terms of research and development aid. In some cases, such experiments have even been shown to be detrimental. I related earlier about one where the distribution of free anti-malarial bed nets killed a local industry.
Second, as Angus Deaton discusses, the data that is obtained in these randomization studies is not informative. The critical issue here is that these experiments are not designed with any theory in mind, thus they do not help us in understanding the underlying mechanisms. They are case studies, applicable only to the very situation they have been used in. This criticism is very similar to the Lucas critique. Unless you put structure in your data, there is nothing useful you can learn from an elasticity in a linear regression with a set of variables which happen to be those available. Add to this that randomization, if poorly performed, gives statistically very poor results.
Third, I have yet to see a study that would indicate anything about the cost effectiveness of a policy or treatment. Studies are all focused on dtermining whether there is a significant impact in a statistical sense, sometimes in an economic sense, but never discusses the cost of the policy. In fact, given the huge cost of these studies, one starts to wonder whether those researchers ever think about scarcity and budget constraints.
What you really want to learn from an experiment is what is generalizable, what can be applied to other situations that differ from the studied one. For this, development economics needs to refocus on theory and the use of theory in its empirical work. Theory can help us understand a surprising amount without needing much data. In fact, in an evironment that is data poor, theory should be the priority, and any quantification should be performed with data-economizing techniques, such as calibration.
A few examples:
Friday, April 10, 2009
Subscribe to:
Post Comments (Atom)
9 comments:
I think the randomization crowd has made a positive contribution to the literature. In particular, they have demonstrated that it is possible to answer some questions that weren’t well answered previously. Randomized experiments have set a new standard for project evaluation, although they can be quite expensive, as you say.
However, Deaton’s critique is also appropriate. These studies don't answer all of the questions that we might want to address, and randomization doesn't magically solve all the identification problems to the extent that we might like.
I am personally a fan of the calibrated models that you mention (and a contributor to the literature), but these too have some drawbacks. While the calibrated models offer perfect identification of causality, they beg the question of validity… Within the models, we understand the causal relationships; but it is not always clear how well the models represent reality.
It seems to me that (catch your breath!) there is a tradeoff between approaching development economics from a theory-based perspective and approaching it from an empirical perspective. I think there is room for both approaches, and I would like to think they are somewhat complementary.
One important issue with the randomizers is that they do case studies with reduced forms. So you end up with one data point, and it cost you a million to get it. I can get much better uses of that research money.
"I think there is room for both approaches, and I would like to think they are somewhat complementary."
While I agree with you 100 percent, the climate is made intolerable by the fact that some of the most prominent randomistas completely and aggressively dismiss any finding that does not come from an RCT.
There is (much) more to development economics than project evaluation, as some of us are interested in learning about whether theoretical models are validated and in knowing about longer chains of causality.
Notice, also, how a lot of the atheoretical, "clever IV" studies spend very little time thinking about potential nonlinear relationships. I am not necessarily advocating exclusively doing structural econometrics, but economics (and social science) will not progress under a monopoly of thought and/or method.
What is the retort of Esther Duflo and the J-PAL clan to this criticism? They must at least be aware of the paper by Angus Deaton and have tried to defend themselves, right?
Their response, I imagine, is that observed data rarely satisfies the selection on observables assumption that is necessary in making causal inference. Instruments are used based upon untestable assumptions. RCTs guarantee selection on observables and provide a clear source of exogeneity that can be leveraged for causal inference. So RCTs give you an internally-valid data point with little theory leading to external validity assumptions. OLS or other strategies may be flawed such that you can't even get internal validity due to confounding.
Randomization could well be effective in those areas where it has been applied and no disrespect to all other methodologies that have been employed in research that has been structured on the platform of economics and development. Not wanting to pry further into what the effects of an underdevelopment would mean for the vast areas that have not been fortunate to be covered by the probabilistic chances that randomization presents, we tend to view its applicability with respect to the unique situations in Sub Saharan African societies.
I believe this was the technique employed by Jeffrey Sachs and his team to Malawi, and the MDG villages that are scattered in various locations in the many underdeveloped and developing regions of the world. Well, much as some can be pointed out as success stories, until its capabilities are broad enough to engage the underdeveloping tendencies inherent in any one particular society (borrowing Amatya Sen's concept), it is still kind of a long way off.
It might be education in one society, it might be infant mortality in another, in Africa, they are as varied as they come and this goes for all the other sub-developed regions of the world too.
The nature of macroeconomic data mentioned and it's seeming ambiguity coupled with the almost non-existent microeconomic data and the assumed insignificance of the nanoeconomic data (which increasingly play significant roles in the perpetuation of underdevelopment in SSA) makes a 21st century economic paradigm not only desirable, but necessary. Areghan IK
I don't disagree with the body of this article just the premise. The introduction is highly misguided. the most important issues in the field of economic development have little to do with investment in human capital. As a corollary, disparities between countries are of minor interest. the Key issue is what causes growth. there is little to no evidence for human capital investment on the aggregate scale having a very significant impact. I purport that the key fact restraining copious growth is the various factors that disincentivise R+D investments. It is obvious from a cursory look at the history of economic growth(i.e. mainly 1870-now) that the big changes are all directly caused by breakthroughs in technology. Steel, steam engines, electricity, telephones, flight,internal combustion engines, automobile engineering, architectural advancements, vaccines,television, antibiotics, plastics, advanced polymers, microwaves, computers,lasers, space flight and satellites, the internet, etc. etc. Every vastly welfare improving phenomena aside from sociological shifts has been a direct result of research and invention. There are all these blind utilitarians out there that don't see what they are missing. human capital arguments are the fly on the wall and R+D type issues are the wrecking ball about to demolish it. Otherwise i think it is a very interesting article.
I wasn't very clear in the beginning of my last post. It's not that the most important issues in development economics are the issues i addessed rather than the ones in the article. Its that the issues of growth in general are vastly more important than explaining why countries that suck at developing do so. to put it this way eventually, assuming human civilization continues for a very long time, The mean wealth in OECD countries will be 1 billion 2009 US dollars. At that point we won't be so worried that there are a bunch of "poor" people in africa struggling to survive on a meager 500k.
Post a Comment