Experimenter bias is not a well-covered topic in the area of experimental economics, either because those who decide to test it don't find anything worth reporting, or those who do attempt to test it struggle to be published. Who knows?

Nevertheless, a 2004 working paper by Alessandro Innocenti and Maria Grazia Pazienza addresses the question of experimenter bias with respect to the gender of the experimenter. They used the trust game with various treatments to check whether the gender of the experimenter changed any of the responses by the subjects. The trust game involves a pairing of two individuals (a sender and a receiver), the sender chooses to 'invest' an amount in the other person (trust element), this amount is then tripled in value and the receiver may then choose to send some money back to the sender (reciprocity element).

There results were as follows:

1. There was no evidence that the gender of the experimenter affected the degree of trust.
   
2. The presence of a female experimenter was correlated with a statistically significant higher degree of reciprocity.
   
3. The effect (of female experimenter reciprocity inflation) occurred across the genders of the subjects, i.e. it was not a 'correspondence' effect of woman-to-woman as some might argue.
   

They had other results, but these are not pertinent to my discussion of experimenter bias.

The main point here is that the fact that the presence of a female experimenter was correlated with increased reciprocity is problematic for experimental economics and requires several pertinent points for its investigation. First, any experiments that might have historically been carried out with women only as experimenters, without variation across the gender of the experimenter could result in over-statement of the reciprocity of individuals. I could not find specific references right now where this could be the case, but it is worth keeping in mind. Second, it begs the question of how we can do an experiment with the gender of the experimenter unknown. Ideally, we would like to be in a world where we could test the relevance of both male or female gender experimenters against some neutral 'non-gender'. Unless we do not have an experimenter present and disembodied instructions being given to the subjects, I do not see how this could be dealt with. Third, it informs future research because it informs us that we must do one of two things: either have male experimenters for whom the results do not seem to be inflated, or have male and female experimenters while controlling for the gender of the experimenter in the experiments. This second option is a bit problematic because it requires us to have larger samples in our experiments, but it seems to me to be the only way that there can be some kind of equitable way of doing this. Fourth, even though there seems to be a correlation of the gender of the experimenter with higher reciprocity, we do not know why this might occur. This requires further research.

It must be said that, personally, I would like to engage in similar research in the future, but using treatments across the race of the experimenter. I think that this would be especially pertinent in South Africa where there might (I insist, might) be differences in responses of individuals when there is variation across the race of the experimenter, holding gender constant of course. What do you think?