Friday, July 17, 2009
Randomized Controlled Trials in the Social Sciences
Posted by Simon Halliday | Friday, July 17, 2009 | Category:
Development,
Microeconomics,
Research
|
There's a debate going on about Randomized Control Trials (RCTs) in Economics. Kicked off by Bill Easterly (in response to recent 'academic' debates in journals about the same topic between Angus Deaton and others). Take a look at Bill Easterly's blog post, and his (requested) comments by Chris Blattman.
Three points are relevant in my view:
1) RCTs need to be replicated, and such replications must be published. This requires that economists and social scientists generally address the institutional inertia about publishing replications. Top level journals need to start publishing 'replication' issues, or have a 'replications' section devoted entirely to verification of previous empirical results. This might be a pipe dream, but I think it is necessary for us to maintain the 'science' in social science. Esther Duflo (pictured above) favours something like this.
2) RCTs are not widely generalizable. They are often local, they are tailored to specific conditions, and the results that economists obtain might be culturally contingent. Hence, an RCT in one area might work because it does not contravene a group's or culture's moral sentiments, whereas in another area a group might have different values and respond differently to the same incentives. Obviously the Haifa Daycare centre is the most famous example of this, but there are many others.
3) People who talk about the 'ethics' of allocating an RCT seem to miss the boat. I am with Blattman entirely on this one: if people can see that allocation of a 'treatment' is transparent and uncorruptible then they will believe that it is ethical. Drawing numbers/names out of hats, or some equivalent public and transparent randomizing mechanism covers this. But the ethics of dealing within RCTs is crucial, i.e., understanding that they are subjects of a social science experiment, but that does not mean you can treat them as chess pieces, they are people and we require their informed consent (John List has controversial views on this topic). Hopefully, sensitive social scientists who engage in such work will adhere to such a dictum.
Hopefully my two cents can help to propagate an informed view of RCTs to those who read this blog.
Three points are relevant in my view:
1) RCTs need to be replicated, and such replications must be published. This requires that economists and social scientists generally address the institutional inertia about publishing replications. Top level journals need to start publishing 'replication' issues, or have a 'replications' section devoted entirely to verification of previous empirical results. This might be a pipe dream, but I think it is necessary for us to maintain the 'science' in social science. Esther Duflo (pictured above) favours something like this.
2) RCTs are not widely generalizable. They are often local, they are tailored to specific conditions, and the results that economists obtain might be culturally contingent. Hence, an RCT in one area might work because it does not contravene a group's or culture's moral sentiments, whereas in another area a group might have different values and respond differently to the same incentives. Obviously the Haifa Daycare centre is the most famous example of this, but there are many others.
3) People who talk about the 'ethics' of allocating an RCT seem to miss the boat. I am with Blattman entirely on this one: if people can see that allocation of a 'treatment' is transparent and uncorruptible then they will believe that it is ethical. Drawing numbers/names out of hats, or some equivalent public and transparent randomizing mechanism covers this. But the ethics of dealing within RCTs is crucial, i.e., understanding that they are subjects of a social science experiment, but that does not mean you can treat them as chess pieces, they are people and we require their informed consent (John List has controversial views on this topic). Hopefully, sensitive social scientists who engage in such work will adhere to such a dictum.
Hopefully my two cents can help to propagate an informed view of RCTs to those who read this blog.
Subscribe to:
Post Comments (Atom)
Currently have 0 comments:
Post a Comment