I've been running economic experiments with undergraduate students at the University of Cape Town.  I've learnt a number of things about doing the experiments since I began.  Much of this stuff was not in the first 'bible' of experimental economics that I read a couple of years back - Daniel Friedman and Shyam Sunder's Experimental Methods: A Primer for Economists.  Nor have I seen it in many articles or other stuff publised about experimental economics, though I may have just forgotten the stuff that I've read. So here are some problems and insights that occurred to me.

1. Recruit more people: Recruit a lot of participants and expect that between 10-30% of those who signed up won't come, you'll reach the higher bound of missing participants more readily if it's raining badly and the participants would rather not leave their warm rooms. So recruit more participants than you think you'll need, that way if there are people who don't arrive, you'll be ok. Yes, you could make mistakes, but rather end up with slightly too many people than too few on a regular basis.  I over-recruited always, still I got burned with slightly too few (3 participants fewer than I'd have liked) on one day when it was raining heavily. 

2. Plan for Free Entry: Don't stick too strongly to your list of registered participants.  Plan for people who have not registered to show up.  I did this in my very first experiment, but I ended up turning two people away who I would have liked to have participate because I didn't prepare sufficiently for enough random arrivals.  Often they can slot into the roles of people who didn't arrive, so they counter the forger/weather/double-booked attrition.  To do this you obviously need to make the venue and time of the experiment public knowledge, which I had attempted to do through leaflets and UCT's online system (I don't know if this should be called foresight or paranoia).   I know that allowing day-of-experiment arrivals may affect your sample because people may have communicated the information through social networks, but when you're playing with university students it's unlikely to have a large effect and I expect it won't have any effect at all given the other constraints you deal with when doing economic and social experiments such as these.

3. Confront Disappointment Carefully: Some students don't like being randomly allocated to roles and don't understand that when everyone has the same probability of being a 'Participant A', 'Participant B', 'Participant C', etc that randomness makes the mechanism 'fair', even if the amounts of money at the end of the day if they had perceived them ex ante might be thought of as 'unfair' ex post.  I had a couple of students (literally two) who emailed me after the experiment because they were annoyed they got less money than other people did. One of them said it wasn't an experiment and that I'd used him.  I reminded him that many other disciplines don't give experimental participants the opportunity to make any money, or if they interview you they may not give any rewards, and when they survey you do may not give any kind of gift.  It seems as though the expectation of money influenced these two students and they got annoyed with me for the fact that others walked away with more than them. Oh well... I had to deal with this, 'in the name of science' (I reassure myself).  If you were wondering, the range of money that participants could walk away with was ZAR0 to ZAR120, roughly USD0 to USD16 in exchange rate terms, but substantially more, about $30 in PPP terms (by this site's estimates).  That said, other students enjoyed participating in the experiment and found it exciting and interesting. I reassured the disappointed students that the outcomes were fair. I made a point of not apolosing though as that would have meant I thought something was wrong - I simply pointed out that the system was fair, they just happened to get a partner with different preferences or maybe they had poor luck. I don't think we should apologise for this, though I could be wrong. 

4. Deputise: Recruiting good research assistants (RAs) was a crucial step for me.  I had recruited a couple of RAs for the first experiment, but not enough to deal with the havoc that can occur when people arrive late and want to play, when trying to ensure all forms are in, trying to stuff envelopes with money at a rapid pace, etc.  Two of my colleagues at UCT ended up helping me during the first experiment, which I appreciated greatly.  So I'd suggest that you hire at least one more RA than you initially think you need. Maybe you're gifted and can tell exactly how much of a problem X will be or Y might be. I wasn't that gifted, so I should have hired at least one more RA.  My first experiment went well, but it could have gone even better or as well as the second and third experiments did when I realised I should dedicate some (of my own) money to getting RAs.  You'll be happy you did it after the fact.  RAs make your life easier, though they do add their own complexities at times.  Also learning to deputise is itself a good skill to have and to develop. 

5. Plan/Play with other people: You may think you have everything covered.  You're probably wrong.  Bounce your ideas, your instructions, your forms, your questionnaires off of as many people as possible, both inside and outside of economics (or whatever your discipline happens to be).  I don't mean "check with your supervisor" obviously you've done that already.  I mean that I tested the instructions, forms, etc on my wife, my parents and my friends before I checked stuff with other economists.  It helped. As did checking the stuff with economists and my useful RAs.  Other people see things you don't.  Use these other people. Crowdsource.  Get other people involved, your ideas probably aren't so great that someone with a bit of insight could see a problem you didn't. 

Continuing my trend of reporting on papers about cooperation, I thought I'd comment on a recent paper by Nancy Buchan and co-authors about human cooperation and globalization. I've argued previously about the role of parochialism in punishment and in theories about the evolution of war and cooperation, today, though, the theme is the extent to which more cosmopolitan countries tend to foster individuals who are more willing to cooperate globally. Sounds intuitive, but how does it work experimentally?

Consider a prisoner's dilemma in the form of a multi-layer public goods game (PGG). A basic PGG works as follows: each player is given some money to contribute to a communal pot, that amount is multiplied by some factor (rate of return), but everyone gets a share of the total minus what they initially contributed. Therefore, I do my 'best' if everyone cooperates fully and contributes all their money, while I free ride and contribute nothing. It's a prisoner's dilemma in action - everyone sees this and they shouldn't cooperate. Consider a similar game, but with two communal pots: one 'local' where only people from my area can contribute to, the amount of which is then doubled and of which I receive 1/4 (3 other people in my group can contribute to it), and one 'global' or 'world' account that I and others can contribute to, the total amount of which is tripled, and of which I receive 1/12 (11 other people can contribute to it) . The tension is the following: the direct return is 0.5 from the local account and 0.25 from the world account. But if everyone contributes a maximal amount to the world account, then we all make off with the most money rather than if we had all defected and not made any contributions. This is also a prisoner's dilemma, but with more layers of complexity.

Buchan et al ran this experiment in 6 countries: the US, Italy, Russia, Argentina, South Africa and Iran, looking specifically at large cities in each of the countries. They made two hypotheses. First, globalization could lead to greater parochialism by emphasizing ethnic, local or national groupings. Conversely, globalization could have a palliative effect on parochialism and perhaps enable and sustain greater tolerance. They indexed a country level globalization index (CGI), an index produced by the University of Warwick. They also constructed an individual level globalization index (IGI) based on questions that each subject answered. The authors used quota sampling methods for gender, age and socioeconomic status (though not for rural-urban, which I comment on later).

Now we come to the reason I'm interested in the research. When the authors ran the experiments, they showed that South Africans ranked 5th out of the 6 countries in the CGI: above Iran, but below Argentina. Considering the experimental results, South Africa also ranked 4th or 5th below Italy, Russia and the United States, but almost exactly equal to Argentina in terms of probability of allocating resources to a global account. Iran came in lowest . Globalization at the individual and country seemed a good predictor of international cooperation. If you're sceptical of the globalization result, the authors also point out that:

"analysis of the CGI along with a host of macroindicators such as the rule of law, generalized trust, per capita income, and norms of civic cooperation shows that the CGI is the only macrovariable that is significantly correlated with mean cooperation rates at the world level." (4140)

Apologies for the light blogging. I fly to South Africa next week and I have a number of deadlines before then (applications for conferences, grants, etc), as well as several for when I arrive in Cape Town: exam questions due for external moderator, tutorials must go up on the web, must finish and upload lecture notes, presenting a seminar a few days after I arrive... I have several half-written research posts that I hope to complete soon. So, apologies, but more research on economics and experiments will come soon.

When you come across a line like this in a paper, you can't help but laugh, "We now discuss and explain the cumulative number of sheep played in all rounds of the game." Yes, subjects played sheep. You may wonder how. I shall attempt to explain.

In three papers based on work in South Africa and Namibia, Bjørn Vollan and, in one paper, his co-author Bernd Hayo investigate several different experiments with the Nama people. They ran trust games, trust games with third party punishment, and common pool resource games with groups of villagers in the Northern Cape Province of South Africa and the southern region of Namibia. Their work was supported by BIOTA, a biodiversity and conservation project.

So what kinds of things did they find out? And how on earth were sheep involved? Well, the Nama people spend substantial amounts of time on subsistence herding of goats and sheep. Consequently, when they played the common pool resource game, the game was framed as though the subjects were exploiting common grazing ground by choosing a number of 'sheep' to 'graze' the resource. The game works like a multi-player prisoner's dilemma where the self-interested thing for a subject to do is to 'defect' by having as many sheep as possible in the hope that no one else will choose lots of sheep, but everyone chooses lots of sheep. The social optimum occurs when people exercise self-restraint and jointly have several sheep, but not too many such that the common resource vanishes. Strangely enough, people often don't play completely self-interestedly, instead they choose something in between the individual optimum and the social optimum (see Cardenas and Carpenter, 2008 and Velez et al, 2006). In Bjørn Vollan's work, we see that the Nama also cooperate to some extent with the option to exploit the resource, and, after several rounds, they also are able to vote on adopting a self-regulating policy of punishment, reward or communication, any of which seem to result in a higher degree of cooperation. The experiments were run with groups of five. In the control, when the subjects were given the choice of between 10 and 90 sheep(in tens), 51.6% of people chose the less cooperative options of 60-90 sheep. When, in the latter rounds, a treatment for reward, punishment, or communication, could be voted on the amount of outright defection decreased to 30.7% of the sample, with 46.5% of the sample choosing the highly cooperative 10, 20 or 30 sheep (everyone choosing 20 would be the social optimum).

So we know that introducing some kind of voted rule works, but does any one rule work better? Does 'buy-in' matter? One problems is that the rankings were different in South Africa and Namibia. In the South African community, punishment resulted in lower numbers of sheep chosen with the choice stabilising at around 4 sheep on average; it worked best of all when more people in the group participating in the experiment voted for it. So a community with 'buy-in' to norms could operate more effectively (interesting enough, in the Hayo and Vollan paper there's a strong correlation between voting for punishment and being a Lutheran, one of the few religious affiliation effects they found in the subjects' behavior). Conversely, in Namibia rewards work much better, though the positive effect of decreasing the number of sheep chosen peter off. Vollan argues that this can be explained by rewards 'crowding out' intrinsic motivations to cooperate (read his paper for the full argument).

So, we know what's happened in the CPR game, but what about the trust game and the trust game with third party punishment? Recall that the trust game is a bargain between two subjects: a Trustor and a Trustee. The Trustor chooses some fraction of an endowment to give to the Trustee, the total amount of which is multipled by 3 (this act is called 'trust' below). The Trustee may then send an amount back to the Trustor. In the third party punishment (TPP) variation, a Third Party is introduced with the power to 'punish' the other players: the Third Party can pay to reduce the payoffs of players whose behavior she did not like, either the Trustor, the Trustee or both. In Vollan's results, and moving from sheep to money, we see that South African Nama trust 20% of their endowment to their partner, whereas Namibian Nama trust 40% of their endowment to their partner. The South African group is at the bottom end of all results of trust games internationally (see Cardenas and Carpenter, 2008, Camerer, 2003). Then, in the second set of experiments examing trust, punishment and relatedness, Vollan finds the results presented in the table below.

The paper presents the aggregated results and I could not dis-aggregate them for South Africa and Namibia as in Vollan's other paper. Nevertheless, the results are instructive. We can see, first, that there are substantially different results between villagers, friends and family. Moreover, introducing punishment substantially increases the likelihood that subjects act 'trustingly' (send money to the Trustee) and 'trustworthily' (send money back to the Trustor). But the effect of punishment differs greatly: villagers behave 48.4% more trustingly, friends behave 44.2% more trustingly, and family only 21.9% more trustingly. The relationship is different for trustworthiness: friends come out on top, behaving 65% more trustworthily, villagers 50% more trustworthily, and family 37% more trustworthily. Obviously these rankings occur because family members behave the most trustingly and trustworthily from the outset, but it goes to show that third party punishment may be its most effective between non-relatives in promoting trusting and trustworthy behavior.

The papers are all fascinating, particularly for a South African interested in doing similar work. I hope that the few results I have shown might have piqued your interest enough for you to go and read the papers. I presented what I perceived as the most interesting results without critiquing Vollan's methods too much. That said, Vollan could have done one or two things to improve his methods. For example, and somewhat pettily, with translation into Afrikaans (the language in which the experiments were conducted), Vollan should have translated and back-translated for consistency. Also, we could be given some additional ethnographic information about the Nama's particular institutional structures of third party punishment, though Vollan does provide information about reciprocity, gift-giving, and trusting/trustworthy behavior. He also finds (2008, 16) that the subject display inequity aversion, so I'd think that given the prevalence literature on this topic it would be worth investigating this idea more and offering some insights in the conclusion after the later results. Overall, they're a good contribution to the experimental economics, resource economics and behavioral economics literature and I thoroughly enjoyed reading them.

References

• Hayo, Bernd & Bjørn Vollan. 2009. "Individual Heterogeneity, Group Interaction, and Co-operative Behaviour: Evidence from a Common-Pool Resource Experiment in South Africa and Namibia." Philipps-Universitt Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
• Vollan, Bjørn. 2008a. "Kinship and friendship in a trust game with third party punishment." Philipps-Universitt Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
• Vollan, B. 2008b. Socio-ecological explanations for crowding-out effects from economic field experiments in southern Africa, Ecological Economics, 67 (4), 560-573 DOI: 10.1016/j.ecolecon.2008.01.015

Are we Africans different to the rest of the world in our giving, punishing and trusting behaviour? Three remarkable economic anthropology studies try to examine this kind of question with several ethnic groups in four countries: the Pimbwe, Sukuma and Kahama in Tanzania, the Maasai of Kenya and the Ju/'hoan Bushmen of Namibia and Botswana. I can't to do any of the papers justice with my short comments, but I thought you might find them interesting nevertheless.

The three papers take quite different approaches in their use of economic experiments. Paciotti and Hadley's paper with the Pimbwe and Sukuma looks at the institutional scope of interactions, that is the extent to which cultural practices and norms may imply differences in the ways people play. They show that the two ethnic groups live differently and respond dramatically differently in the experiments. The Sukuma are agro-pastoralists with pre-existing norms of within and between village cooperation, and justice institutions (sungusungu) that punish individuals for norm infringement. Their play in the ultimatum game was more generous than the Pimbwe, with very few rejections (most likely because the offers were fair or hyper-fair). Contrastingly, the Pimbwe do not have many institutions for between village cooperation, or any third party justice institutions and their grievances are often settled with personal violence. Their results are consequently unsurprising - much lower offers on average, with substantially lower offers to those of another village. The remarkable thing here is that even the Sukuma's offers to residents of another village were higher than the Pimbwe's offers to residents in their own village. Also, the Pimbwe rejected offers (which some call punishment) substantially more than the Sukuma. The authors therefore show that even within small geographical distances, differences in institutional backgrounds alter how people behave.

Lee Cronk's study of the Maasai cultural norm of osotua fascinated me even more. Osotua is an often reciprocal relationship between male Maasai who call each other isotuatin. Osotua bonds isotuatin to each other so that they should provide for each other in times of great need, normally requiring assistance with food or gifts of livestock, or even revenge killing. So Cronk decided to see how Maasai subjects played the trust game in two settings: the first without any cultural frame, and the second where it was framed as 'an osotua game'. As you'd expect they played the game quite differently given the frame. In the control (no frame) and considering all transfers the subjects gave 35.3% vs. 28.2% in the framed condition. Considering player 1 (the 'trustor'), the transfers differ 38% (unframed) and 30.8% (unframed), though not statistically significantly so, and transfers by player 2 (the 'trustee') back to player 1 also differ: 32.5% (unframed) vs 25.5% (framed). But only in the framed condition did a strong correlation existed between the amounts given and the amounts returned show through, indicating the reciprocal nature of the osotua. Also, Cronk proposes, the lower amounts given in the osotua frame probably reflect the sense that osotua gifts are only given to assuage great need, and normally are not great amounts. Cronk argues forcibly that anthropologists and economists need to act carefully when they construct games within specific tribal and ethnographic contexts, investigating the norms that exist within a specific context, and ensuring that they tailor their studies accordingly, being careful of when they encounter norms that may alter the data that they find.

Finally, Polly Wiessner's research into the Ju/'hoan bushmen looked at their behavior in the dictator game and the ultimatum game, after which Wiessner examined their behavior in everyday life to see whether experimental behavior and behavior outside the experiment were consistent. In the dictator game, the average offer was 20%, and in the ultimatum game the average offer was 16% with 4% refusals. These averages are the some of the lowest in the world (see Henrich et al, 2006, Barr et al 2009). Notwithstanding these low within game offers, outside the game the Ju/'hoansi share tobacco, pool food resources, ostracize those who had infringed social norms (those they ostracized possibly stole a goat) and act compassionately towards those who behaved unwisely with the money from the experiments (some went to the town, got drunk the so-called "fault of the beer"). As per experimental protocols, the subjects behavior was anonymous. Wiessner was asked repeatedly during the experiments if she was lying about this because the concept seemed quite foreign to the subjects whose behavior is normally socially embedded. Wiessner's work reinforces how researchers need to align laboratory and experimental protocols with the everyday lives of the people involved; though many of us are involved in anonymous market economies the distances that separate us from those who produce the goods we produce are immense, this is not always the case. Moreover, being able to understand the spillovers of anonymous behavior to social embedded behavior and the converse can enlighten the use of experiments in urban and rural, market-integrated and non-market-integrated societies.

The papers serve as a reminder to economists (though how many economists read anthropology journals I do not know) that their work must take cognisance of cultural and institutional structures, of the frames that they introduce with experiments (think of harambee with Jean Ensminger's work), and of the parallelism between experimental behavior and behavior in parallel situations outside of the experiment. In this way, pairing laboratory experiments with field experiments and good ethnography could provide a better way to do things in future. Or so I hope.

References:
Cronk, Lee. 2007. "The Influence of Cultural Framing on play in the trust game: a Maasai example." Evolution and Human Behavior 28(5):352-358.
Paciotti, Brian & Craig Hadley. 2003. "The Ultimatum Game in Southwestern Tanzania: Ethnic Variation and Institutional Scope." Current Anthropology 44(3):427-432.
Wiessner, P. (2009). Experimental Games and Games of Life among the Ju/’hoan Bushmen Current Anthropology, 50 (1), 133-138 DOI: 10.1086/595622