Jonah Lehrer has a recent piece in the WSJ in which he comments on several books on decision-making as a lead up to the release of his upcoming book How We Decide. The problem I wanted to highlight with his article was a reference to the book by Daniel Kahneman, Paul Slovic and Amos Tversky, Judgment Under Uncertainty. Lehrer comments that, "in experiment after experiment, the psychologists demonstrated that, unlike the hypothetical consumers in economics textbooks, real people don't treat losses and gains equivalently."

This is factually incorrect. They did not run experiments, they administered questionnaires. They asked people how they would react as if they were to face a lottery over gains and as if they were to face a lottery over losses. They did not actually run the experiments to test whether people would act in the ways that they described in their seminal (1979) paper, 'Prospect Theory: An Analysis of Decision Under Risk'. To quote Kahneman and Tversky (1979, 264), "The demonstrations are based on the responses of students and university faculty to hypothetical choice problems." Again, not an experiment.

Subsequently, a number of their theories have been validated in the lab, but for a lot of their most famous results they did not run experiments, unless administering questionnaires has suddenly become experimental. Yes, in later papers they did run experiments, and, yes, for certain other research questions they ran experiments, but for the things that Lehrer comments on this did not, especially not prior to 1982. Even as late as 1992 some of Tversky's work was based only on questionnaires (by my count of the papers of his I know), similarly one of their most famous papers with Richard Thaler (1984) was based on questionnaires.

Note that for this I am not saying that questionnaires are useless, I just wanted to clarify that they are not experiments and thus that the norms for other economic experiments, i.e. actually using the monetary incentives involved, were not held to.

Anyway, the books to which he refers are all good. I have read the Ariely, some of the Thaler and some of the Kahneman, Slovic and Tversky. I just wanted to clarify that one point.