Causal inference, moral hazard, and the challenges of social science 

 News that two clinical trials in Africa have been halted because the preliminary results were so strong that it was considered unethical to continue them has received major play in the media ( New York Times ,  Guardian ,  Washington Post ).  The reason: the experimental treatment was male circumcision and the outcome of interest was the risk of female-to-male transmission of HIV.  This is a topic that has been discussed previously in the Applied Statistics workshop (see posts  here  and  here ).  The two studies suggest that circumcision reduces the probability of transmission by about 50%, which is similar to an earlier randomized trial in South Africa (and, it should be noted, the estimated effect is also consistent with the results from a number of observational studies, see this  NIAID FAQ  for more details on the studies).  In short, the evidence seems overwhelming at this point that, from a biomedical perspective, circumcision is effective at reducing transmission. 


 Is the same true from a policy perspective?  In other words, would a policy promoting circumcision reduce the number of new HIV cases?  The answer to that question is much less obvious, the concern being that the men who were circumcised would engage in riskier behavior given their newfound knowledge.  This is a classic moral hazard problem; the people implementing the policy cannot control the actions taken by the treated individuals.  Indeed, the researchers behind the study were falling all over themselves to emphasize the need for continued prevention measures.  Despite this, it seems likely to me that one of the effects of the study (as opposed to the effect of the treatment) is going to be an  increase   in HIV transmission, at least at the margin, among the male subpopulation that is already circumcised.   

 This study thus highlights a couple of issues that face us as social science.  First, the scientific quantity of interest (does circumcision reduce the risk of HIV transmission) need not be, and often isn't, the policy quantity of interest (will circumcision reduce the number of new HIV cases).  Second, unlike our colleagues in the natural sciences, we do have to worry that the behavior of our subjects (broadly defined) will be influenced by the results of our research.  A biologist doesn't have to worry that the dolphins she is studying are reading  Marine Mammal Science  (although to the extent that the modal number of times that a political science article is cited is still  zero , we may not have to worry about our subjects reading the results of our research either!).  From my perspective, the possibility of feedback - that behavior will change in response to research, in ways that could either reinforce or mitigate the conclusions that we draw - is one of the key characteristics that distinguish the social sciences from the natural sciences, a distinction that seems underappreciated and that makes our jobs as researchers substantially harder.