Wednesday, June 10, 2009
Was there a Hawthorne Effect?
Posted by Simon Halliday | Wednesday, June 10, 2009 | Category:
experimental economics,
Microeconomics
|
An article in this week's Economist covers a recent paper by Steve Levitt and John List where they investigate the data from the original factory studies that elicited the so-called 'Hawthorne Effect'. The basic idea of the Hawthorne Effect was that being studied directly affected the productivity of the women in the factory. It turns out the effects may not have been so strong, if it even existed at all. Levitt and List found that the light intensity was changed on a Monday. The original researchers then compared Monday's productivity with Saturday's productivity, and unsurprisingly there was an increase. What they didn't do was compare Monday's productivity with the lights changed to any other Monday. Levitt and List found a systematic decrease in productivity as the week progressed which had nothing to do with being experimented upon.I plan to read the research paper after my exams finish. I will comment on it more comprehensively then. Though there may not have been a Hawthorne Effect, demand effects or demand characteristics (as they are called in psychology) are fairly well-documented with higher quality data and better research protocols than were adhered to in the Hawthorne studies. Nevertheless, that the 'Hawthorne Effect' didn't exist as we believed is a pretty big finding. Well done Levitt and List.
Subscribe to:
Post Comments (Atom)
Well... the study reported on simply cannot show that the Hawthorne Effect does not exist. At most it shows the original study didn't provide evidence for the Hawthorne Effect. As I understand it, there is plenty of other evidence for the effect.
Yes, Mike, I agree. This is why I commented on the more general existence of demand effects and demand characteristics which are well-documented. The interesting thing for me is that the the study that inspired later research actually did not, itself, have the effect that they claim it did.
It's actually not that uncommon for famous experiments to turn out to be flawed on subsequent analysis. Eddington's "confirmation" of general relativity in 1919, for example, was likely no such thing. His methods were actually incapable of distinguishing between Newton and Einstein's predictions regarding gravitational lensing. The history of science is littered with such cases.
Agreed Mike. What it reinforces for me is the phenomenal importance of the following:
1) Replicating studies and journals that publish the results of the replications.
2) Free availability of data to ensure that re-analysis of the same data produces the same results, and, as above, these re-assessments are published in credible journals.
One of the problems in contemporary social science I believe is that there aren't sufficient spaces just to publish replications and re-analyses.
Totally agreed. But it's not just a social science issue -- it's a science issue.
I think there is a real case to be made that journals should require the online publication of all raw data. That would help a lot of replication (published or not) and post-publication peer review.
a lot with not of...